Unable to see Watchers in the Sentinl Watcher Table

I am using ES version 6.2.3, Kibana version 6.2.2

Issue description
Even though it says that the watcher is successfully created, I still do not see the same in the watcher table

Reproducing the issue step-by-step

I was expecting a watcher table something like this

Can somebody please help me out?

Could you confirm if you are using any authentication mechanism? Also grab the Kibana logs as you save the watcher, they should reveal some sort of error response while attempting to store data.

Hi,
Thanks for the response.
I created a watcher named - test. Here is the log. There is no authentication mechanism.
"{"type":"log","@timestamp":"2018-06-20T13:49:18Z","tags":["warning"],"pid":15274,"kibanaVersion":"6.2.2","nodes":[{"version":"6.2.3","http":{"publish
_address":"<IP_ADDRESS>:6968"},"ip":"<IP_ADDRESS>"},{"version":"6.2.3","http":{"publish_address":"<IP_ADDRESS>:6968"},"ip":"<IP_ADDRESS>"},{"ve
rsion":"6.2.3","http":{"publish_address":"<IP_ADDRESS>:6968"},"ip":"<IP_ADDRESS>"},{"version":"6.2.3","http":{"publish_address":"<IP_ADDRESS>:6
968"},"ip":"<IP_ADDRESS>"},{"version":"6.2.3","http":{"publish_address":"<IP_ADDRESS>:6968"},"ip":"<IP_ADDRESS>"},{"version":"6.2.3","http":{"p
ublish_address":"<IP_ADDRESS>:6968"},"ip":"<IP_ADDRESS>"},{"version":"6.2.3","http":{"publish_address":"<IP_ADDRESS>:6968"},"ip":"<IP_ADDRESS>
3"},{"version":"6.2.3","http":{"publish_address":"<IP_ADDRESS>:6968"},"ip":"<IP_ADDRESS>"}],"message":"You're running Kibana 6.2.2 with some diff
erent versions of Elasticsearch. Update Kibana or Elasticsearch to the same version to prevent compatibility issues: v6.2.3 @ <IP_ADDRESS>:6968 (192
.168.10.69), v6.2.3 @ <IP_ADDRESS>:6968 (<IP_ADDRESS>), v6.2.3 @ <IP_ADDRESS>:6968 (<IP_ADDRESS>), v6.2.3 @ <IP_ADDRESS>:6968 (<IP_ADDRESS>
16), v6.2.3 @ <IP_ADDRESS>:6968 (<IP_ADDRESS>), v6.2.3 @ <IP_ADDRESS>:6968 (<IP_ADDRESS>), v6.2.3 @ <IP_ADDRESS>:6968 (<IP_ADDRESS>), v6.
2.3 @ <IP_ADDRESS>:6968 (<IP_ADDRESS>)"}
{"type":"response","@timestamp":"2018-06-20T13:49:20Z","tags":[],"pid":15274,"method":"get","statusCode":200,"req":{"url":"/api/saved_objects/?type=sentinl-watcher&per_page=10&search=%22test%22&search_fields=title&fields=title&page=1","method":"get","headers":{"host":"<IP_ADDRESS>:5601","connection":"keep-alive","accept":"application/json, text/plain, /","kbn-version":"6.2.2","user-agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36","referer":"http://<IP_ADDRESS>:5601/app/sentinl","accept-encoding":"gzip, deflate","accept-language":"en-GB,en-US;q=0.9,en;q=0.8"},"remoteAddress":"<IP_ADDRESS>","userAgent":"<IP_ADDRESS>","referer":"http://<IP_ADDRESS>:5601/app/sentinl"},"res":{"statusCode":200,"responseTime":21,"contentLength":9},"message":"GET /api/saved_objects/?type=sentinl-watcher&per_page=10&search=%22test%22&search_fields=title&fields=title&page=1 200 21ms - 9.0B"}
{"type":"log","@timestamp":"2018-06-20T13:49:20Z","tags":["warning"],"pid":15274,"kibanaVersion":"6.2.2","nodes":[{"version":"6.2.3","http":{"publish_address":"<IP_ADDRESS>:6968"},"ip":"<IP_ADDRESS>"},{"version":"6.2.3","http":{"publish_address":"<IP_ADDRESS>:6968"},"ip":"<IP_ADDRESS>"},{"version":"6.2.3","http":{"publish_address":"<IP_ADDRESS>:6968"},"ip":"<IP_ADDRESS>"},{"version":"6.2.3","http":{"publish_address":"<IP_ADDRESS>:6968"},"ip":"<IP_ADDRESS>"},{"version":"6.2.3","http":{"publish_address":"<IP_ADDRESS>:6968"},"ip":"<IP_ADDRESS>"},{"version":"6.2.3","http":{"publish_address":"<IP_ADDRESS>:6968"},"ip":"<IP_ADDRESS>"},{"version":"6.2.3","http":{"publish_address":"<IP_ADDRESS>:6968"},"ip":"<IP_ADDRESS>"},{"version":"6.2.3","http":{"publish_address":"<IP_ADDRESS>:6968"},"ip":"<IP_ADDRESS>"}],"message":"You're running Kibana 6.2.2 with some different versions of Elasticsearch. Update Kibana or Elasticsearch to the same version to prevent compatibility issues: v6.2.3 @ <IP_ADDRESS>:6968 (<IP_ADDRESS>), v6.2.3 @ <IP_ADDRESS>:6968 (<IP_ADDRESS>), v6.2.3 @ <IP_ADDRESS>:6968 (<IP_ADDRESS>), v6.2.3 @ <IP_ADDRESS>:6968 (<IP_ADDRESS>), v6.2.3 @ <IP_ADDRESS>:6968 (<IP_ADDRESS>), v6.2.3 @ <IP_ADDRESS>:6968 (<IP_ADDRESS>), v6.2.3 @ <IP_ADDRESS>:6968 (<IP_ADDRESS>), v6.2.3 @ <IP_ADDRESS>:6968 (<IP_ADDRESS>)"}
Best
Aashish

I see no useful info in the Kibana log you posted. Do you see the created watcher in the .kibana index?
curl -XGET localhost:9200/.kibana/doc/_search?pretty

You can also specify a field value to narrow search
curl -XGET localhost:9200/.kibana/_search?q=sentinl-watcher.title:telecom30

Do you see any error in your browser dev console while adding a watcher?

This is dev Console Logs
""http://<IP_ADDRESS>:5601/app/sentinl"},"res":{"statusCode":200,"responseTime":13,"contentLength":9},"message":"GET /api/saved_objects/?type=senti
nl-script&search=condition*&per_page=50&page=1&search_fields=title%5E3&search_fields=description 200 13ms - 9.0B"}
{"type":"response","@timestamp":"2018-06-21T07:22:08Z","tags":[],"pid":15274,"method":"get","statusCode":200,"req":{"url":"/api/saved_objects/?type=
sentinl-script&search=transform*&per_page=50&page=1&search_fields=title%5E3&search_fields=description","method":"get","headers":{"host":"192.168.10.
121:5601","connection":"keep-alive","accept":"application/json, text/plain, /","kbn-version":"6.2.2","user-agent":"Mozilla/5.0 (X11; Linux x86_64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.170 Safari/537.36 OPR/53.0.2907.99","referer":"http://<IP_ADDRESS>:5601/app/sentinl","acc
ept-encoding":"gzip, deflate","accept-language":"en-GB,en-US;q=0.9,en;q=0.8"},"remoteAddress":"<IP_ADDRESS>","userAgent":"<IP_ADDRESS>","referer":
"http://<IP_ADDRESS>:5601/app/sentinl"},"res":{"statusCode":200,"responseTime":8,"contentLength":9},"message":"GET /api/saved_objects/?type=sentin
l-script&search=transform*&per_page=50&page=1&search_fields=title%5E3&search_fields=description 200 8ms - 9.0B"}
Unhandled rejection [export_exception] Exception when closing export bulk :: {"path":"/_xpack/monitoring/_bulk","query":{"system_id":"kibana","syste
m_api_version":"6","interval":"10000ms"},"body":"{"index":{"_type":"kibana_stats"}}\n{"concurrent_connections":1303,"os":{"load":{"1m"
:0.5361328125,"5m":0
.49365234375,"15m":0.341796875},"memory":{"total_in_bytes":25267707904,"free_in_bytes":2928754688,"used_in_bytes":22338953216},"uptime_in_millis":13376074000},"process":{"event_loop_delay":10001.011863708496,"memory":{"heap":{"total_in_bytes":179527680,"used_in_bytes":155893264,"size_limit":1501560832},"resident_set_size_in_bytes":250347520},"uptime_in_millis":64416429},"requests":{"disconnects":0,"total":659,"status_codes":{"200":248,"304":397,"404":13}},"response_times":{"average":4777,"max":4777},"timestamp":"2018-06-21T07:22:05.330Z","kibana":{"uuid":"192694e9-faaf-4114-8ab7-60ae14082c6c","name":"S25","index":".kibana_main","host":"S25","transport_address":"<IP_ADDRESS>:5601","version":"6.2.2","snapshot":false,"status":"green"},"usage":{"index":".kibana_main","dashboard":{"total":33},"visualization":{"total":308},"search":{"total":75},"index_pattern":{"total":47},"graph_workspace":{"total":0},"timelion_sheet":{"total":0},"xpack":{"reporting":{"available":true,"enabled":true,"browser_type":"phantom","_all":0,"csv":{"available":true,"total":0},"printable_pdf":{"available":false,"total":0}}}}}\n","statusCode":500,"response":"{"took":2,"errors":true,"error":{"type":"export_exception","reason":"Exception when closing export bulk","caused_by":{"type":"export_exception","reason":"failed to flush export bulks","caused_by":{"type":"export_exception","reason":"failed to flush export bulk [default_local]","caused_by":{"type":"illegal_state_exception","reason":"There are no ingest nodes in this cluster, unable to forward request to an ingest node."}}}}}"}
at respond (/home/user/kibana-6.2.2-linux-x86_64/node_modules/elasticsearch/src/lib/transport.js:295:15)
at checkRespForFailure (/home/user/kibana-6.2.2-linux-x86_64/node_modules/elasticsearch/src/lib/transport.js:254:7)
at HttpConnector. (/home/user/kibana-6.2.2-linux-x86_64/node_modules/elasticsearch/src/lib/connectors/http.js:159:7)
at IncomingMessage.bound (/home/user/kibana-6.2.2-linux-x86_64/node_modules/elasticsearch/node_modules/lodash/dist/lodash.js:729:21)
at emitNone (events.js:91:20)
at IncomingMessage.emit (events.js:185:7)
at endReadableNT (_stream_readable.js:974:12)
at _combinedTickCallback (internal/process/next_tick.js:80:11)"

this is output of the Curl Command
curl -XGET localhost:9200/.kibana/_search?q=sentinl-watcher.title:telecom30
"{"took":1,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}"

There are no ingest nodes in this cluster, unable to forward request to an ingest node.

This seems to be the core issue, and its a cluster one apparently? does this message appear in the Elasticsearch logs too?

Hi Lorenzo,
We are using sentinl with ES2, there also we do not have any ingest node and we never faced any issue.
There is an index called Watcher in ES2 and all the watchers created are stored in that index. Where does Senitnl for ES6 saves the watchers?

Hi Loranzo,
We are using Logstash. Is it pre-requisite for Sentinl to use ES ingest node?

Hi Sergi,
There is no sentinl watcher in .kibana index. Does it stores the watchers in .kibana? Sentinl 2 used to store watchers in an index called "Watcher"

It is working fine for below configuration:
Kibana 6.2.2
ES: 6.2.2
Sentinl: 6.2.2
But not working for:
Kibana 6.2.2
ES: 6.2.3
Sentinl: 6.2.2

There is no sentinl watcher in .kibana index. Does it stores the watchers in .kibana? Sentinl 2 used to store watchers in an index called "Watcher"

Yes, Sentinl currently stores only in .kibana index.
@aashish051218 please show me you kibana.yml, sentinl config part.

Hi Sergii,
Here are the files.
Kibana.yml.txt
Sentinl json.txt

Hi Sergii,
Did you see an anomaly?
Best
Aashish

Hi @aashish051218 ,
If you watcher is getting created successfully but still not able to see in the watcher list in sentinl, then you need to increase the "results" in kibana.yml.
Let me know if you still face the same issue.

I also face this problem.Can not see the setted watcher lists in sentinl.But the watcher job work normal.
kibana sentinl error:
Sentinl Alarms: An HTTP request has failed to connect. Please check if the Kibana server is running and that your browser has a working connection, or contact your system administrator.

Hi Sergii,
Here are the files.
Kibana.yml.txt
Sentinl json.txt

Hi @aashish051218 ,
I see that there is no results field set in your kibana.yml file. Please set the results field in sentinl section of your kibana.yml and then restart kibana.
Note: set the results values around 1000, if you believe that there will be almost 1000 sentinl alerts.
Reference: https://sentinl.readthedocs.io/en/latest/Config-Example/
The results value show how many sentinl alerts who want to see in Sentinl dashboard.
Let me know if you are still unable to see the alert.

Hi Sergii,
Here are the files.
Kibana.yml.txt
Sentinl json.txt

Hi @aashish051218 ,
I see that there is no results field set in your kibana.yml file. Please set the results field in sentinl section of your kibana.yml and then restart kibana.
Note: set the results values around 1000, if you believe that there will be almost 1000 sentinl alerts.
Reference: https://sentinl.readthedocs.io/en/latest/Config-Example/
The results value show how many sentinl alerts who want to see in Sentinl dashboard.
Let me know if you are still unable to see the alert.

I set parameters follows:but unable to see the sentinl watcher

job(but the jobs work nomal to alert mails.)

sentinl:
settings:
email:
active: true
host: 10.42..
ssl: false
report:
active: true
tmp_path: /tmp/

Hi @aashish051218 ,
You are not following my instruction. Please refer the last comment from me. You will understand the issue. Also, you can refer the attached snapshot for your reference. Make sure you restart Kibana after making the changes in kibana.ym

Hi @aashish051218 ,
You are not following my instruction. Please refer the last comment from me. You will understand the issue. Also, you can refer the attached snapshot for your reference. Make sure you restart Kibana after making the changes in kibana.ym

Hello,
I set parameters follow:
sentinl:
es:
host: localhost
port: 9200
timefield: '@timestamp'
default_index: watcher
type: sentinl-watcher
alarm_index: watcher_alarms
alarm_type: sentinl-alarm
script_type: sentinl-script
sentinl:
history: 20
results: 50
scriptResults: 50
settings:
email:
active: false
user: ***
password: ***
host: wksowa.wistron.com
ssl: true
timeout: 10000  # mail server connection timeout
But : unable to see the setted alert job lists in sentinl&I also can not set new alert job.
had the error:(the kibana server work ok.and the old sentinl job work nomal too)
Sentinl Watchers: An HTTP request has failed to connect. Please check if the Kibana server is running and that your browser has a working connection, or contact your system administrator.

Hi @aashish051218 ,
Apology for the confusion.
I just wanted you to add "results" field in your kibana.yml. (Do not add any other things in your kibana file). Kindly revert it and just add "results" field in the existing kibana file which was working fine previously(like below)
=====
#Sentinl configuration
sentinl:
settings:
email:
active: true
user<EMAIL_ADDRESS>password: xxxx
host: secure.emailsrvr.com
host: ticket.or.net
  ssl: true
report:
  active: true
  executable_path: '/usr/bin/google-chrome'
sentinl:
  history: 100
  results: 500

=====
Make sure you restart kibana after the change.

Hi @aashish051218 ,
Apology for the confusion.
I just wanted you to add "results" field in your kibana.yml. (Do not add any other things in your kibana file). Kindly revert it and just add "results" field in the existing kibana file which was working fine previously(like below)
#Sentinl configuration sentinl: settings: email: active: true user<EMAIL_ADDRESS>password: xxxx host: secure.emailsrvr.com ssl: true report: active: true executable_path: '/usr/bin/google-chrome' sentinl: history: 100 results: 500
Make sure you restart kibana after the change.

i had setted this:
sentinl:
settings:
email:
active: true
host: wksowa.wistron.com
ssl: true
timeout: 10000  # mail server connection timeout
report:
active: true
tmp_path: /tmp/
sentinl:
history: 100
results: 500
Still unable see job lists.(can still see it when it's just built but can't see it after a few months)


Map keys

I want to make a response keys mapping before requests are made and before response is assigned to model. Is this possible without overriding methods like onFetchSuccess()?
One thing that could work would having different mutators for responses and requests.

There are two methods you can use to handle this. They're both preserved with semver so it's safe to override them. They are getSaveData and update. However, I think we should rename update to something like useSavedData or applySave or setSaveData to be consistent with the language of getSaveData.
In the meantime, you can do this:
getSaveData() {
    let data = super.getSaveData(); // Don't need to, can use this.attributes instead

    // Adjust the data, map keys etc.

    return data;
}

update(data) {
    // Reverse what the server responded with to match your data structure.
}

While this works, it shouldn't be necessary. Your controller should define and transform a serialised representation of your model so that it can accept and receives the same schema.
What's your use case here?

This is necessary once I don't have control on backend API and only on frontend application.

Oh right, that makes a lot of sense. I didn't consider the situation where you might not have control over the backend. Let me know if overriding those methods don't cover your use case. 👍

It works. It would be great to support this feature in a mutators way..

Update method doesn't work for fetch for example. I'm trying to make a login method, on getRequest().send() success I'm calling onFetchSuccess but it does not pass by update method. I can't change it to assign method because it's used in both ways (frontend to API and API to frontend). Do you have any suggestions?


ENH: Basic anonymization

This modifies the way we were anonymizing meas_info objects to mainly permit time shifting dates.
It also adds an executable for anonymizing raw fif files from the command line.
The goal was to match the current plans in mne_cpp (https://github.com/mne-tools/mne-cpp/pull/393) which is still under development. hence the [WIP] tag.
closes  #6844 and #4982
@LorenzE

these tests pass locally on my mac. Not sure how best to debug?

@bloyl I can't replicate when I use EDT for my current time zone. When I switch to European time (Travis is German) it breaks. So you should be able to replicate by changing your computer's apparent time zone, which means something is not being done in absolute time somewhere.

Pushed some RST changes to go from this to this. Let's merge once CIs come back happy -- thanks @bloyl !

thx @bloyl and @LorenzE for your insights


⚠ Turnos 24x7 has degraded performance

In 9df86e2, Turnos 24x7 (https://reservas-dev.alternativasinteligentes.com/) experienced degraded performance:

HTTP code: 200
Response time: 11850 ms

Resolved: Turnos 24x7 performance has improved in d98983f.


scorer & calc_score

Please make sure that this is a feature request.
System information

Hypernets version (you are using):
Are you willing to contribute it (Yes/No):

Describe the feature and the current behavior/state.
I found that the user-defined evaluation metric function could not be set in the hypernets experiment, and calc_score() contained few metrics, such as no mape.
Will this change the current api? How?
Any Other info.

Is it possible to make the following modifications?
https://github.com/DataCanvasIO/HyperTS/blob/main/hyperts/utils/metrics.py


Periodic stuck pulls require restarts to fix until they start to happen again

Hi,
We've deployed a caching registry using version 2.7.1. The problem we're having is that pulls periodically hangs (they get stuck on a random layer and do not move forward) and the only way to fix, temporarily, is to restart the registry cache. How can we troubleshoot this or what other information can we provide to help find the issue here ?
Thanks

@dg424 could you please provide logs from the registry from around the time of the hangs?
What kind of cache are you using?

Hi,
This is a docker container image registry. This is our current configuration:
/ # cat /etc/docker/registry/config.yml version: 0.1 log:   fields:     service: registry storage:   cache:     blobdescriptor: inmemory   filesystem:     rootdirectory: /var/lib/registry http:   addr: :5000   headers:     X-Content-Type-Options: [nosniff] health:   storagedriver:     enabled: true     interval: 10s     threshold: 3
We do not have logging enabled as per here - https://docs.docker.com/registry/configuration/#log. We've been restarting the registry as the problem happens to get things back online, so I think we might have lost the docker logs. Is there anything we can set in the configuration to prepare for the next time this happens ? Is there any known issue with pulls (or pushes) getting stuck after a period of time with the only recourse being to restart to fix ?
Thanks

We do not have logging enabled as per here

I believe you can only disable the access log, regular logging should still be active and that should be enough

We've been restarting the registry as the problem happens to get things back online, so I think we might have lost the docker logs. Is there anything we can set in the configuration to prepare for the next time this happens ?

I think so, if you encounter this issue again, please capture the logs before restarting

Is there any known issue with pulls (or pushes) getting stuck after a period of time with the only recourse being to restart to fix ?

Not to my knowledge. Are you using a plain local filesystem, or something like an NFS share or iSCSI that communicates over a network?


Error when trying to use JOOQ's Result to ResultSet

When using JOOQ and using .fetch().intoResultSet()  instead of .fetchResultSet() I get an exception:
java.lang.IllegalStateException: No date format specified 	at org.simpleflatmapper.converter.impl.time.JavaTimeHelper.getDateTimeFormatters(JavaTimeHelper.java:56) 	at org.simpleflatmapper.converter.impl.time.AbstractMultiFormatContextualConverterFactory.newConverter(AbstractMultiFormatContextualConverterFactory.java:20) 	at org.simpleflatmapper.converter.ConverterService.findConverter(ConverterService.java:90) 	at org.simpleflatmapper.map.fieldmapper.ConstantSourceFieldMapperFactoryImpl.lookForAlternativeGetter(ConstantSourceFieldMapperFactoryImpl.java:272) 	at org.simpleflatmapper.map.fieldmapper.ConstantSourceFieldMapperFactoryImpl.getGetterFromSource(ConstantSourceFieldMapperFactoryImpl.java:249) 	at org.simpleflatmapper.map.fieldmapper.ConstantSourceFieldMapperFactoryImpl.newFieldMapper(ConstantSourceFieldMapperFactoryImpl.java:195) 	at org.simpleflatmapper.map.mapper.DefaultConstantSourceMapperBuilder.newFieldMapper(DefaultConstantSourceMapperBuilder.java:920) 	at org.simpleflatmapper.map.mapper.DefaultConstantSourceMapperBuilder$PropertyFieldMeta.targetFieldMapper(DefaultConstantSourceMapperBuilder.java:757) 	at org.simpleflatmapper.map.mapper.DefaultConstantSourceMapperBuilder.targetFieldMappers(DefaultConstantSourceMapperBuilder.java:649)...
If I use the normal .fetchResultSet() way everything works.

The column types are timestamptz (PostgreSQL) and the destination types are OffsetDateTime (the records and pojos are generated being generated by JOOQ).

Don't now if it helps but the sfm version im using is 6.0.8

when using fetch().intoResultSet() jooq is creating a mock resultset with metadata return has
sqlType : 12 or VARCHAR
getColumnClassName : "OffsetDateTime"
getColumnTypeName : "timestamp with time zone"
the other result set
sqlType : 93 or VARCHAR
getColumnClassName : "Timestamp"
getColumnTypeName : "timestamptz"
which we could use to deal with it

Is it something i can fix in my code or do I have to wait for you to fix it?

No I don't have a twitter account. I prefer fetch().intoResultSet() because it handles everything for me. From the JOOQ docs:

Instead of operating on a JDBC ResultSet holding an open resource from your database, you can also let jOOQ's org.jooq.Result wrap itself in a java.sql.ResultSet. The advantage of this is that the so-created ResultSet has no open connection to the database. It is a completely in-memory ResultSet:

I found about this because HikariCP was complaining about connection leaks due to me using the ResultSet without closing it. So I found about this wrapping which would solve the problem entirely but that exception prevents me from using this solution. I am just using the other way and handling the result set closing and what not but it is not as nice or clean.

6.0.9 is now available in maven central with the fix.

Thank you very much.

np, could you just confirm it worked? when you can?
Thanks for the report!

Can confirm it is working. If I find any problem during my testing in the next few days I'll let you know. Thanks again.


Inspiration (Humble Request)

Please take inspiration from breathe app available on play store

Sure, what Platform are you on? I'm on telegram you can find me in the group chat https://t.me/naoxio


deps - bump eth deps

This fixes resolvers missing the isLink method

Hmm actually I realize the resolvers are still not spec compliant


New Memory backend, replace ETS?

We seem to have a few issues with ETS backends, that are hard to make any better. Maybe a custom map-based storage server as the new default?

@ShaneKilkelly Can you share what are the issues with ETS backend?
I've noticed it's guarded by single GenServer process for both reads and writes - maybe allowing concurrent ETS reads from any process could improve the performance and make the ETS backend production-ready?

@ShaneKilkelly just coming across this project. We are using ExRated right now on fontawesome.com and we are taking a look at switching to hammer.
And I second what @teamon is saying. I believe the performance issues are because of the synchronous nature of GenServer. ETS by itself is blistering fast but by wrapping it with atomic handle_call it slows everything down to essentially a single thread. We did this same thing on fontawesome.com for a bit until we read this article.
@ShaneKilkelly are you interested in a PR that takes the ETS backend and speeds things up for the critical path calls?

Yup, I'd love to see that kind of PR! Thanks!

@ShaneKilkelly take a look at #41 and see what you think

this is merged and solved


Update fluxnet workflow

We still have to fully move Fluxnet workflow to use new data loader system.

Closed by #42 and #43 afaik


ROX-21039: Coordinate Scanner V4 / Clairify Interactions

Description
This PR introduces the following changes:

Sorts scanner integrations so that scannerv4 scanners occurs before clairify scanners
When Scanner V4 feature enabled:

Create the default Scanner V4 integration on Central startup
Prevent manual create (via API) of Scanner V4 integrations
Prevent manual delete (via API) of Scanner V4 integrations
Prevent changing (via API) an integration type to/from Scanner V4
Prevent a clairify scan from overwriting a scannerv4 scan

When Scanner V4 feature disabled:

Delete Scanner V4 integrations on Central startup

Builds on top of:

https://github.com/stackrox/stackrox/pull/8869

Checklist

[x] Investigated and inspected CI test results
[X] Unit test and regression tests added
[X] Evaluated and added CHANGELOG entry if required
[X] Determined and documented upgrade steps
[X] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

Testing Performed
Here I tell how I validated my change
On top of unit tests, various scenarios were executed per design / requirements:
Confirm integration created on startup if ROX_SCANNER_V4_ENABLED = true
$ k set env deploy/central ROX_SCANNER_V4_ENABLED="true"

$ da tail central | grep -i scannerv4
imageintegration/datastore: 2023/12/15 03:47:28.457420 singleton.go:111: Info: Upserting default Scanner V4 integration "ScannerV4" (a87471e6-9678-4e66-8348-91e302b6de07)
pkg/scanners/scannerv4: 2023/12/15 03:47:28.719896 scannerv4.go:74: Debug: Creating Scanner V4 with name [ScannerV4] indexer address [scanner-v4-indexer.stackrox.svc:8443], matcher address [scanner-v4-matcher.stackrox.svc:8443], num concurrent scans [30]

Attempt create scannerv4 integration via API (expect error)
$ cat > payload.json << EOF
{
    "name": "ScannerV4",
    "type": "scannerv4",
    "categories": [
   	 "SCANNER"
    ],
    "scannerV4": {
    }
}
EOF

$ curl -ks -H "Authorization: Bearer $TOKEN" -X POST "https://$ACSHOST/v1/imageintegrations" -H "Content-Type: application/json" -d @payload.json | jq
{
  "error": "scanner V4 integrations cannot be manually created: invalid arguments",
  "code": 3,
  "message": "scanner V4 integrations cannot be manually created: invalid arguments",
  "details": []
}

Attempt delete scannerv4 integration via API (expect error)
$ curl -ks -H "Authorization: Bearer $TOKEN" -X DELETE "https://$ACSHOST/v1/imageintegrations/a87471e6-9678-4e66-8348-91e302b6de07" | jq
{
  "error": "scanner V4 integrations cannot be deleted: invalid arguments",
  "code": 3,
  "message": "scanner V4 integrations cannot be deleted: invalid arguments",
  "details": []
}

Attempt modification of scannerv4 integration via API (expect success)
$ cat > payload.json << EOF
{
    "id": "a87471e6-9678-4e66-8348-91e302b6de07",
    "name": "ScannerV4",
    "type": "scannerv4",
    "categories": [
    "SCANNER"
    ],
    "scannerV4": {
    "numConcurrentScans": 5,
    "indexerEndpoint": "https://indexer.example.com",
    "matcherEndpoint": "https://matcher.example.com"
    }
}
EOF

$ curl -ks -H "Authorization: Bearer $TOKEN" -X PUT "https://$ACSHOST/v1/imageintegrations/a87471e6-9678-4e66-8348-91e302b6de07" -H "Content-Type: application/json" -d @payload.json
{}

$ da tail central | grep -i scannerv4
pkg/scanners/scannerv4: 2023/12/15 03:38:21.355375 scannerv4.go:158: Warn: ScannerV4 - Returning FAKE 'success' to Test
pkg/scanners/scannerv4: 2023/12/15 03:38:21.368646 scannerv4.go:74: Debug: Creating Scanner V4 with name [ScannerV4] indexer address [https://indexer.example.com], matcher address [https://matcher.example.com], num concurrent scans [5]

Confirm integration persists with modifications after central restart
$ k delete pods -n stackrox -lapp=central

$ da tail central | grep -i scannerv4
pkg/scanners/scannerv4: 2023/12/15 03:41:06.072835 scannerv4.go:74: Debug: Creating Scanner V4 with name [ScannerV4] indexer address [https://indexer.example.com], matcher address [https://matcher.example.com], num concurrent scans [5]

$ curl -ks -H "Authorization: Bearer $TOKEN" -X GET "https://$ACSHOST/v1/imageintegrations/a87471e6-9678-4e66-8348-91e302b6de07" | jq
{
  "id": "a87471e6-9678-4e66-8348-91e302b6de07",
  "name": "ScannerV4",
  "type": "scannerv4",
  "categories": [
    "SCANNER"
  ],
  "scannerV4": {
    "numConcurrentScans": 5,
    "indexerEndpoint": "https://indexer.example.com",
    "matcherEndpoint": "https://matcher.example.com"
  },
  "autogenerated": false,
  "clusterId": "",
  "skipTestIntegration": false,
  "source": null
}

Verify can reset integration back to default values
$ cat > payload.json << EOF
{
    "id": "a87471e6-9678-4e66-8348-91e302b6de07",
    "name": "ScannerV4",
    "type": "scannerv4",
    "categories": [
    "SCANNER"
    ],
    "scannerV4": {
    }
}
EOF

$ curl -ks -H "Authorization: Bearer $TOKEN" -X PUT "https://$ACSHOST/v1/imageintegrations/a87471e6-9678-4e66-8348-91e302b6de07" -H "Content-Type: application/json" -d @payload.json
{}

$ da tail central | grep -i scannerv4
pkg/scanners/scannerv4: 2023/12/15 03:44:14.061942 scannerv4.go:158: Warn: ScannerV4 - Returning FAKE 'success' to Test
pkg/scanners/scannerv4: 2023/12/15 03:44:14.076825 scannerv4.go:74: Debug: Creating Scanner V4 with name [ScannerV4] indexer address [scanner-v4-indexer.stackrox.svc:8443], matcher address [scanner-v4-matcher.stackrox.svc:8443], num concurrent scans [30]

Confirm integration deleted on startup if ROX_SCANNER_V4_ENABLED = false
$ k set env deploy/central ROX_SCANNER_V4_ENABLED="false"

$ da tail central | grep -i scannerv4
imageintegration/datastore: 2023/12/15 03:46:36.984696 singleton.go:136: Info: Deleted Scanner V4 integration "ScannerV4" (a87471e6-9678-4e66-8348-91e302b6de07)

$ curl -ks -H "Authorization: Bearer $TOKEN" -X GET "https://$ACSHOST/v1/imageintegrations/a87471e6-9678-4e66-8348-91e302b6de07" | jq
{
  "error": "image integration a87471e6-9678-4e66-8348-91e302b6de07 not found: not found",
  "code": 5,
  "message": "image integration a87471e6-9678-4e66-8348-91e302b6de07 not found: not found",
  "details": []
}

Ensure cannot change an integration's type to scanner v4
$ cat > payload.json << EOF
{
  "name": "test",
  "type": "quay",
  "categories": [
    "SCANNER"
  ],
  "quay": {
    "endpoint": "https://example.com"
  },
  "skipTestIntegration": true
}
EOF

$ curl -ks -H "Authorization: Bearer $TOKEN" -X POST "https://$ACSHOST/v1/imageintegrations" -H "Content-Type: application/json" -d @payload.json | jq
{
  "id": "1b8a6440-8328-41e7-b1c2-532c39be76d7",
  "name": "test",
  "type": "quay",
  "categories": [
    "SCANNER"
  ],
  "quay": {
    "endpoint": "https://example.com",
    "oauthToken": "",
    "insecure": false,
    "registryRobotCredentials": null
  },
  "autogenerated": false,
  "clusterId": "",
  "skipTestIntegration": true,
  "source": null
}

$ cat > payload.json << EOF
{
  "name": "test",
  "type": "scannerv4",
  "categories": [
    "SCANNER"
  ],
  "scannerV4": {
  },
  "skipTestIntegration": true
}
EOF

$ curl -ks -H "Authorization: Bearer $TOKEN" -X PUT "https://$ACSHOST/v1/imageintegrations/1b8a6440-8328-41e7-b1c2-532c39be76d7" -d @payload.json | jq
{
  "error": "cannot change integration type to/from scanner V4: invalid arguments",
  "code": 3,
  "message": "cannot change integration type to/from scanner V4: invalid arguments",
  "details": []
}

Ensure cannot change a scanner v4 integration to another type
$ cat > payload.json << EOF
{
  "name": "ScannerV4",
  "type": "clairify",
  "categories": [
    "SCANNER"
  ],
  "clairify": {
    "endpoint":"https://example.com"
  }
}
EOF

$ curl -ks -H "Authorization: Bearer $TOKEN" -X PUT "https://$ACSHOST/v1/imageintegrations/a87471e6-9678-4e66-8348-91e302b6de07" -d @payload.json | jq
{
  "error": "cannot change integration type to/from scanner V4: invalid arguments",
  "code": 3,
  "message": "cannot change integration type to/from scanner V4: invalid arguments",
  "details": []
}

Ensure scanner v2 save does not override scannerv4 save, and ensure that when not using --force the DB returns the scannerv4 scan when ROX_SCANNER_V4_ENABLED = true
$ rctl image scan --image=docker.io/jenkins/jenkins:2.414.1 --force | jq '.scan.dataSource'
{
  "id": "a87471e6-9678-4e66-8348-91e302b6de07",
  "name": "ScannerV4"
}

$ da tail central | grep -i scannerv4
pkg/scanners/scannerv4: 2023/12/15 03:49:03.552643 scannerv4.go:129: Debug: Vuln report received for "docker.io/jenkins/jenkins:2.414.1" (hash "/v4/containerimage/sha256:066c3e3eb2c7534b6f2146163c0ffe7e8f7a9d0ae64ead405a6f4ebc3f56326d"): 1 dists, 401 envs, 401 pkgs, 2 repos, 74 pkg vulns, 178 vulns

$ k scale deploy/scanner-v4-indexer --replicas 0

$ rctl image scan --image=docker.io/jenkins/jenkins:2.414.1 --force | jq '.scan.dataSource'
{
  "id": "169b0d3f-8277-4900-bbce-1127077defae",
  "name": "Stackrox Scanner"
}

$ da tail central | grep -i -E "scannerv4|scanner v4"
image/service: 2023/12/15 03:52:01.062683 service_impl.go:221: Warn: Cannot overwrite Scanner V4 scan already in DB with Clairify scan and cannot calculate risk scores {"image": "docker.io/jenkins/jenkins:2.414.1"}

# observe a new save is created / overwritten when forcing the scan
$ rctl image scan --image=docker.io/jenkins/jenkins:2.414.1 | jq '.scan | .scanTime, .dataSource'
"2023-12-15T03:50:53.886946913Z"
{
  "id": "a87471e6-9678-4e66-8348-91e302b6de07",
  "name": "ScannerV4"
}

$ k scale deploy/scanner-v4-indexer --replicas 1

$ rctl image scan --image=docker.io/jenkins/jenkins:2.414.1 --force | jq '.scan | .scanTime, .dataSource'
"2023-12-15T03:59:09.825303369Z"
{
  "id": "a87471e6-9678-4e66-8348-91e302b6de07",
  "name": "ScannerV4"
}

$ rctl image scan --image=docker.io/jenkins/jenkins:2.414.1 | jq '.scan | .scanTime, .dataSource'
"2023-12-15T03:59:09.825303369Z"
{
  "id": "a87471e6-9678-4e66-8348-91e302b6de07",
  "name": "ScannerV4"
}

Ensure scanner v2 save CAN override a scanner v4 save when ROX_SCANNER_V4_ENABLED=false
$ k set env deploy/central ROX_SCANNER_V4_ENABLED="false"

$ rctl image scan --image=docker.io/jenkins/jenkins:2.414.1 | jq '.scan.dataSource'
{
  "id": "a87471e6-9678-4e66-8348-91e302b6de07",
  "name": "ScannerV4"
}

$ rctl image scan --image=docker.io/jenkins/jenkins:2.414.1 --force | jq '.scan.dataSource'
{
  "id": "169b0d3f-8277-4900-bbce-1127077defae",
  "name": "Stackrox Scanner"
}

$ rctl image scan --image=docker.io/jenkins/jenkins:2.414.1 | jq '.scan.dataSource'
{
  "id": "169b0d3f-8277-4900-bbce-1127077defae",
  "name": "Stackrox Scanner"
}

Create image watch and confirm image saved to DB:

pkg/scanners/scannerv4: 2023/12/15 04:15:53.243708 scannerv4.go:129: Debug: Vuln report received for "docker.io/library/ubuntu:bionic-20191010" (hash "/v4/containerimage/sha256:3e83eca7870ee14a03b8026660e71ba761e6919b6982fb920d10254688a363d4"): 1 dists, 89 envs, 89 pkgs, 0 repos, 56 pkg vulns, 304 vulns

Reminder for reviewers
In addition to reviewing code here, reviewers must also review testing and request further testing in case the
performed one does not seem sufficient. As a reviewer, you must not approve the change until you understand the
performed testing and you are satisfied with it.

@dhaus67 @RTann - FYI, I moved the logic to prevent overwriting a scanner V4 save with a V2 save from the image service (central/image/service/service_impl.go) to the risk manager (central/risk/manager/manager.go).
Did this because other areas of ACS (ie: detection service, development service, etc.) could also overwrite the save, the risk manager is the common / convergence point for saving an image to the DB.

FYI, will be revisiting the image upgrade type change logic again - it appears the UI calls the /v1/imageintegrations/test/updated to test 'new' integrations which breaks logic.

/retest

/test gke-scanner-v4-tests


(B4) Register as baker on node dashboard

confirmed:B4
address:4YY5717adVgJLhx6V6D4NLcHw6vCqLB4utBveJ6b7fsR5DPio6
baker:703

confirmed:B4
address:4YY5717adVgJLhx6V6D4NLcHw6vCqLB4utBveJ6b7fsR5DPio6
baker:703


Add TPESampler for the tutorial paper

Contributor Agreements
Please read the contributor agreements and if you agree, please click the checkbox below.

[x] I agree to the contributor agreements.

[!TIP]
Please follow the Quick TODO list to smoothly merge your PR.

Motivation

This sampler is to give a more stable implementation of the TPE tutorial paper.
Description of the changes

Add a more parameterized TPESampler to reproduce the TPE tutorial paper

TODO List towards PR Merge

[x] Copy ./template/ to create your package
[x] Replace <COPYRIGHT HOLDER> in LICENSE of your package with your name
[x] Fill out README.md in your package
[x] Add import statements of your function or class names to be used in __init__.py
[x] Apply the formatter based on the tips in README.md
[x] Check whether your module works as intended based on the tips in README.md

@eukaryo Could you check this PR?

I replaced the assignee with @y0z as @eukaryo is busy now.

@nabenabe0928
Could you add an Abstract section to README.md? (cf. https://github.com/optuna/optunahub-registry/pull/130#event-13786300608)
It will help attract potential users to your package.

@y0z I added abstract!

@y0z done!


Add runscope and pagerduty for hub.blockstack.org and gaia.blockstack.org

Add a pager duty alert for hub.blockstack.org

Also document the post format to Gaia Hub during this process


how can i run test.py

when i run test.py, an error occurred，as follows，
usage: _jb_nosetest_runner.py [-h] [--trained_model TRAINED_MODEL]
[--save_folder SAVE_FOLDER]
[--visual_threshold VISUAL_THRESHOLD]
[--cuda CUDA]
_jb_nosetest_runner.py: error: unrecognized arguments: /home/zjh/ssd.pytorch-master-nan/test.py
Process finished with exit code 2
Empty test suite.
Can someone help me?tks

I have the similar error. when I run the eval.py, like this
usage: _jb_pytest_runner.py [-h] [--trained_model TRAINED_MODEL]
[--save_folder SAVE_FOLDER]
[--confidence_threshold CONFIDENCE_THRESHOLD]
[--top_k TOP_K] [--cuda CUDA]
[--voc_root VOC_ROOT] [--cleanup CLEANUP]
_jb_pytest_runner.py: error: unrecognized arguments: F:/pychqarmtestdemo/ssd.pytorch-master/eval.py
Do you know how to solve it?

I have the same matter,could you tell me how to solve it ?

I have the similar error. when I run the eval.py, like this
usage: _jb_pytest_runner.py [-h] [--trained_model TRAINED_MODEL]
[--save_folder SAVE_FOLDER]
[--confidence_threshold CONFIDENCE_THRESHOLD]
[--top_k TOP_K] [--cuda CUDA]
[--voc_root VOC_ROOT] [--cleanup CLEANUP]
_jb_pytest_runner.py: error: unrecognized arguments: F:/pychqarmtestdemo/ssd.pytorch-master/eval.py
Do you know how to solve it?

I have the same matter,could you tell me how to solve it ?


can you change what CVE database it uses?

hello,
is it possible to change the website where it gets the CVE's from? for example if I have this site
https://cve.circl.lu/ would I be able to change to get from that site?

@jlozadad
It would be very useful if your database had information on whether exploit code was published


Cosmos DB occasionally throws Invalid URL error when using items.create

Package Name: @azure/cosmos
Package Version: ^4.0.0
Operating system: Windows 11 Home
[x] nodejs

version: v20.12.2

[ ] browser

name/version: Google Chrome 126.0.6478.127

[ ] typescript

version: N/A

Is the bug related to documentation in

[ ] README.md
[ ] source code documentation
[ ] SDK API docs on https://docs.microsoft.com

Describe the bug
When using the items.create method repeatedly to upload data, after a while and more often than not (roughly about 80% of the time), the app will fail with an ECONNRESET or ETIMEDOUT error code. Upon further investigation the error it appears to come from the @azure/core-rest-pipeline/dist/commonjs/util/sanitizer.js file on line 104:
node:internal/url:796
    this.#updateContext(bindingUrl.parse(input, base));
                                   ^

TypeError: Invalid URL
    at new URL (node:internal/url:796:36)
    at Sanitizer.sanitizeUrl (C:\Users\joelw\Development\zaiix\zaiix-scraper\zapk-search-sample\node_modules\@azure\core-rest-pipeline\dist\commonjs\util\sanitizer.js:104:21)
    at IncomingMessage.<anonymous> (C:\Users\joelw\Development\zaiix\zaiix-scraper\zapk-search-sample\node_modules\@azure\core-rest-pipeline\dist\commonjs\util\sanitizer.js:72:29)
    at JSON.stringify (<anonymous>)
    at Sanitizer.sanitize (C:\Users\joelw\Development\zaiix\zaiix-scraper\zapk-search-sample\node_modules\@azure\core-rest-pipeline\dist\commonjs\util\sanitizer.js:63:21)
    at [nodejs.util.inspect.custom] (C:\Users\joelw\Development\zaiix\zaiix-scraper\zapk-search-sample\node_modules\@azure\core-rest-pipeline\dist\commonjs\restError.js:27:64)
    at formatValue (node:internal/util/inspect:805:19)
    at inspect (node:internal/util/inspect:364:10)
    at formatWithOptionsInternal (node:internal/util/inspect:2298:40)
    at formatWithOptions (node:internal/util/inspect:2160:10) {
  code: 'ERR_INVALID_URL',
  input: ''
}

And it appears that whatever this sanitizer function is, is failing to parse an empty string. I don't know why an empty string would be parsed in the first place? But either this function should be fixed to handle this, or the items.create method needs to be fixed to not parse a blank string.
Can anyone explain why this is happening?
Expected behavior
Item gets added to the container.

Additional info: This problem appears to occur when many requests are made at once (potentially looking at some kind of accidental DOS attack on my own database, however this is what the rate limiting should account for..

Thanks for report this issue @joelwright-dev! Very likely this would be addressed by PR https://github.com/Azure/azure-sdk-for-js/pull/30151

The issue was that when we log request and responses, we try to remove sensitive informations.  Our sanitizer wasn't handling url property of empty string properly so when there's some property or nested property with url name and empty string value this error was thrown.
We just released a fix to the TypeError: Invalid URL issue in @azure/core-rest-pipeline 1.16.2.  Please refresh your dependencies to get the latest version.


Reportium Download methods are throwing errors for local execution

Need to skip the reportium download methods in case of local execution.
The end user will need to make sure that the 'remote.server' key is kept empty for the local execution.

Fixed in version 1.20


update "Searching" DSL

the old "Searching" syntax for ES 6.* is wrong, so I commit the right one

update "Searching"

I have a similar problem. I haven't solved it yet. I tried your method and it really solved it.

Thanks for the PR, @JamieLybo 👍


Update reference to serverless-workflow in tekton task to point to main

Update reference to serverless-workflow in tekton task to point to main

@jordigilh Or should a new 1.3.x branch be create in the serverless-workflows repo and then we should point to it?

@jordigilh Or should a new 1.3.x branch be create in the serverless-workflows repo and then we should point to it?

I created v1.3.x branch in serverless-workflows repo, now pointing to it as it is the latest


[Question] Lifecycle/istio is showing similar configurator for servish mesh.

Current Behavior

Following katakoda scenario from here: https://layer5.io/learn/service-mesh-labs/working-with-meshery-and-istio

I am seeing a lots of similar accordions doing similar things/setting up similar behaviour for istio. Is it normal?
Screenshots/Logs

Environment

Host OS: Linux
Platform: Docker
Client  v0.5.66  8aa1d17c
Server  v0.5.66  8aa1d17c

Contributor Guides and Resources

🛠 Meshery Build & Release Strategy
📚 Instructions for contributing to documentation

Meshery documentation site and source

🎨 Wireframes and designs for Meshery UI in Figma
🙋🏾🙋🏼 Questions: Layer5 Discussion Forum and Layer5 Community Slack

Makes sense. Please see https://layer5io.slack.com/archives/CLVT4TSG4/p1634039583368200?thread_ts=1634039447.367400&cid=CLVT4TSG4

Need to verify this behaviour in the lifecycle is still exists.

@Abhishek-kumar09 I faced this issue a couple of days back, I think it still persists

Fixed


fix: disable custom window refresh method on Windows 11

The useCustomWindowRefreshMethod option doesn't appear to be needed on Windows 11, as Microsoft seems to have fixed the issue (finally). This should automatically stop the feature from kicking in, regardless of the options set by the developer.
I'm unable to test this PR completely, as I have no idea how to get the example running. yarn can't seem to find my Python installation, and I don't use yarn to bother to looking into it. So you should double check this this is working correctly before merging.

First of all, thank you for your contribution.
Currently, my college entrance exam is just around the corner, so I don't think I can check this commitment immediately. I'm really sorry about this. I'll check again as soon as possible.

@Seo-Rii No worries, take your time. Good luck with your exam! 👍

LGTM


Corrected link for .NET SDK for DAPR

Description
Corrected broken link for .NET SDK for DAPR
Adding the SDK for .Net from Github since not available in docs.dapr.io
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: #600
Checklist
No Code changes
Only updated link for broken url

@AaronCrawfis - can you review? What's the link that's supposed to go here?

We will have the .NET docs in the docs website soon, hopefully next week.


Failed to build x86_64-apple-darwin binary on ARM silicon

hi, when I compile the rust-ffmpeg x86 program on Apple M2 Pro macOS 15.1, I get an error
Cargo.toml
ffmpeg-next = {git = "https://github.com/zmwangx/rust-ffmpeg.git", branch = "master"}

error message:
error: failed to run custom build command for `ffmpeg-sys-next v7.1.0`

Caused by:
  process didn't exit successfully: `/Users/bochao/Work/projects/SearchIn/src-tauri/target/release/build/ffmpeg-sys-next-201ad7040f4dd9cc/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=LIBAVUTIL_NO_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_x86_64-apple-darwin
  cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_x86_64_apple_darwin
  cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_ALLOW_CROSS
  cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-apple-darwin
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_apple_darwin
  cargo:rerun-if-env-changed=TARGET_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-apple-darwin
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_apple_darwin
  cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_SYSROOT_DIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR

  --- stderr
  thread 'main' panicked at /Users/bochao/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ffmpeg-sys-next-7.1.0/build.rs:795:14:
  called `Result::unwrap()` on an `Err` value: pkg-config has not been configured to support cross-compilation.

  Install a sysroot for the target platform and configure it via
  PKG_CONFIG_SYSROOT_DIR and PKG_CONFIG_PATH, or install a
  cross-compiling wrapper for pkg-config and set it via
  PKG_CONFIG environment variable.
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
failed to build x86_64-apple-darwin binary: failed to build app
    Error failed to build x86_64-apple-darwin binary: failed to build app
 ELIFECYCLE  Command failed with exit code 1.

Did you solve it?

Did you solve it?

@caobug Try this

Install Rosetta2 via terminal using:

/usr/sbin/softwareupdate --install-rosetta --agree-to-license

Once Rosetta 2 is installed, install Homebrew for the x86_64 chip.

arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

Then install the x86_64 version of the ffmpeg libraries.

arch -x86_64 /usr/local/bin/brew install ffmpeg

arch -x86_64 /usr/local/bin/brew install pkg-config

Set PKG_CONFIG env （pkg-config bin）

# using your pkg-config path
export PKG_CONFIG=/usr/local/Cellar/pkg-config/0.29.2_3/bin/pkg-config

build rust-ffmpeg again

Finally, I removed the build feature in ffmpeg-next.features and manually configured the pkg-config environment variable.
# build
targets=(aarch64-apple-ios x86_64-apple-ios aarch64-apple-ios-sim)
for target in "${targets[@]}"
do
  export PKG_CONFIG_LIBDIR=~/documents/cross-compile/$target/pkgconfig
  export PKG_CONFIG_SYSROOT_DIR=/
  cargo build --target $target $RUN_ENV || exit
done


Samplesheetmodeli

eline sağlık
On Wed, 7 Aug 2024 at 17:44, Sinan Erdem @.***> wrote:

Merged #8 https://github.com/gen-era/canvas/pull/8 into development.
—
Reply to this email directly, view it on GitHub
https://github.com/gen-era/canvas/pull/8#event-13793682711, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AKIZI5K2BUT2DZDS5CSPKODZQIXEVAVCNFSM6AAAAABMESH6ZOVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJTG44TGNRYGI3TCMI
.
You are receiving this because you authored the thread.Message ID:
@.***>


Forum is ready!

Please give me some feedback, and then I officially publish it!
https://forum.indie.host/
Cheers :)

Cool! But sign up doesn't work, the email doesn't arrive.

Ah, spam folder of course.

Cool! All seems to be working fine. Are backups working?

You got it in your spam folder? This is not normal, what is your email provider?
And yes, backups are working :)


Linux_android flutter_gallery__start_up is 2.54% flaky

The post-submit test builder Linux_android flutter_gallery__start_up had a flaky ratio 2.54% for the past 15 days, which is above our 2.00% threshold.
One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android flutter_gallery__start_up/588/overview
Commit: 18ed542e910266c4e59113674a457faa0639845b
Flaky build: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android flutter_gallery__start_up/588/overview (check step run flutter_gallery__start_up for both failed and successful executions)
Please follow https://github.com/flutter/flutter/wiki/Reducing-Test-Flakiness#fixing-flaky-tests to fix the flakiness and enable the test back after validating the fix (internal dashboard to validate: go/flutter_test_flakiness).

Maybe DevTools is failing to start? The only flake noted by the flake bot is following https://github.com/flutter/flutter/pull/90894, which changed the way that DevTools is launched. @keyonghan were there any other actual flakes? Mostly what I see in the builder history is infra failures. /cc @kenzieschmoll
[flutter_gallery__start_up] [STDOUT] stdout: [        ] An Observatory debugger and profiler on Moto G 4 is available at: http://<IP_ADDRESS>:34541/PfYLuM9MIz0=/
[flutter_gallery__start_up] [STDOUT] stdout: [   +3 ms] executing: /opt/s/w/ir/cache/android/platform-tools/adb -s ZY22465W9K forward --list
[flutter_gallery__start_up] [STDOUT] stdout: [   +2 ms] Exit code 0 from: /opt/s/w/ir/cache/android/platform-tools/adb -s ZY22465W9K forward --list
[flutter_gallery__start_up] [STDOUT] stdout: [        ] ZY22465W9K tcp:35233 tcp:45188
[flutter_gallery__start_up] [STDOUT] stdout: [   +1 ms] executing: /opt/s/w/ir/cache/android/platform-tools/adb -s ZY22465W9K forward --remove tcp:35233
[flutter_gallery__start_up] [STDOUT] stderr:
[flutter_gallery__start_up] [STDOUT] stderr: NoSuchMethodError: The getter 'uri' was called on null.
[flutter_gallery__start_up] [STDOUT] stderr: Receiver: null
[flutter_gallery__start_up] [STDOUT] stderr: Tried calling: uri
[flutter_gallery__start_up] [STDOUT] stderr: #0      Object.noSuchMethod (dart:core-patch/object_patch.dart:68:5)
[flutter_gallery__start_up] [STDOUT] stderr: #1      ResidentRunner.printDebuggerList (package:flutter_tools/src/resident_runner.dart:1382:47)
[flutter_gallery__start_up] [STDOUT] stderr: #2      FlutterResidentDevtoolsHandler.serveAndAnnounceDevTools (package:flutter_tools/src/resident_devtools_handler.dart:96:23)
[flutter_gallery__start_up] [STDOUT] stderr: <asynchronous suspension>
[flutter_gallery__start_up] [STDOUT] stderr:
[flutter_gallery__start_up] [STDOUT] stdout: [  +13 ms] ensureAnalyticsSent: 0ms
[flutter_gallery__start_up] [STDOUT] stdout: [        ] Running shutdown hooks
[flutter_gallery__start_up] [STDOUT] stdout: [        ] Shutdown hooks complete
[flutter_gallery__start_up] [STDOUT] stdout: [   +2 ms] "flutter run" took 16,291ms.
[flutter_gallery__start_up] [STDOUT] stdout: [   +1 ms] exiting with code 1
[flutter_gallery__start_up] [STDOUT] "/opt/s/w/ir/x/w/recipe_cleanup/tmprVWMq9/flutter sdk/bin/flutter" exit code: 1

Some other flake builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android flutter_gallery__start_up/581/overview on commit: https://github.com/flutter/flutter/commit/dc59bc54448dba7cee945879094ea12e054157d5
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android flutter_gallery__start_up/578/overview on commit: https://github.com/flutter/flutter/commit/d94c18d2b919388efb8a84ad72814a240192824a

Current flaky ratio for the past 15 days is 8.60%.
One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android flutter_gallery__start_up/638/overview
Commit: 45ffcc646d070a2b256d26d3b6eb70fe2a938ec1
Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android flutter_gallery__start_up/638/overview

https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android flutter_gallery__start_up/638/overview flaked due to:
error: device 'ZY223KBHPZ' not found

Is there a stale package_config.json under that subdirectory somewhere? @christopherfujino is this a case where flutter update-packages is needed? Looks like the pubspec has the right version constraint: https://github.com/flutter/flutter/blob/master/dev/devicelab/pubspec.yaml#L6.

flutter update-packages fixed it. I ran this locally but could not reproduce failures.

@zanderso @keyonghan I am not sure how to fix this since I cannot reproduce locally. Do either of you have ideas for a fix?

Will the bot re-file this if we close it?

The bot will hold for 15 days for a recently closed flaky bug, and will re-file if the flaky percentage > 2% starting on the third week.

Not sure that I have any more info to add to this bug. Assigning back to @zanderso.

@godofredoc were you able to find out anything about linux-2?

We have an infra issue for linux2 in the past two weeks: https://github.com/flutter/flutter/issues/92429. As this bug was filed like a month ago, not sure how they correlate here. Let me check the current flake status.

The last 200 runs have been successful with a few infra failures because the lack of capacity in staging env.
https://ci.chromium.org/p/flutter/builders/staging/Linux_android flutter_gallery__start_up?limit=200
Linux-2 had an inflated battery and was replaced with a new phone on 10/29. I believe we can close this bug and reopen if it happens again.


Log JS errors

Setup with Sentry
Errors are viewable here:
https://sentry.io/organizations/karrlab/issues/?project=5264018

The free version of Sentry only supports 1 user. I switched to AirBrake.


Why 6.0.0-alpha.2 not on npm ?

I have the same question as well.

Because we were waiting for a few issues to resolve but most people seems to be on vacation.


How to unload Lora from sdxl to achieve dynamic switching of Lora?

Are there any example codes available for uninstalling Lora? Similar to diffusers unload_lora_weights

What script or inference environment do you use?
If you use networks.lora.py from another script, there are backup_weights, pre_calculation and restore_weights methods.

Thank you. It works.


Installation incompatible with window 11

Caused by:
process didn't exit successfully: C:\Users\Dell\AppData\Local\Temp\cargo-installbxk00K\release\build\lz4-sys-04cc6fce4401a058\build-script-build (exit code: 1)
--- stdout
TARGET = Some("x86_64-pc-windows-gnu")
HOST = Some("x86_64-pc-windows-gnu")
CC_x86_64-pc-windows-gnu = None
CC_x86_64_pc_windows_gnu = None
HOST_CC = None
CC = None
CFLAGS_x86_64-pc-windows-gnu = None
CFLAGS_x86_64_pc_windows_gnu = None
HOST_CFLAGS = None
CFLAGS = None
CRATE_CC_NO_DEFAULTS = None
DEBUG = Some("false")
CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
running: "gcc.exe" "-O3" "-ffunction-sections" "-fdata-sections" "-m64" "-Wall" "-Wextra" "-o" "C:\Users\Dell\AppData\Local\Temp\cargo-installbxk00K\release\build\lz4-sys-b1f17b681a26919f\out\liblz4/lib/lz4.o" "-c" "liblz4/lib/lz4.c"
running: "gcc.exe" "-O3" "-ffunction-sections" "-fdata-sections" "-m64" "-Wall" "-Wextra" "-o" "C:\Users\Dell\AppData\Local\Temp\cargo-installbxk00K\release\build\lz4-sys-b1f17b681a26919f\out\liblz4/lib/lz4frame.o" "-c" "liblz4/lib/lz4frame.c"
--- stderr
error occurred: Failed to find tool. Is gcc.exe installed? (see https://github.com/alexcrichton/cc-rs#compile-time-requirements for help)
error: failed to run custom build command for zstd-sys v2.0.1+zstd.1.5.2
Caused by:
process didn't exit successfully: C:\Users\Dell\AppData\Local\Temp\cargo-installbxk00K\release\build\zstd-sys-6835696c1a4a47d2\build-script-build (exit code: 1)
--- stdout
TARGET = Some("x86_64-pc-windows-gnu")
HOST = Some("x86_64-pc-windows-gnu")
CC_x86_64-pc-windows-gnu = None
CC_x86_64_pc_windows_gnu = None
HOST_CC = None
CC = None
CFLAGS_x86_64-pc-windows-gnu = None
CFLAGS_x86_64_pc_windows_gnu = None
HOST_CFLAGS = None
CFLAGS = None
CRATE_CC_NO_DEFAULTS = None
DEBUG = Some("false")
CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
running: "gcc.exe" "-O3" "-ffunction-sections" "-fdata-sections" "-m64" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-DZSTD_DISABLE_ASM=" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-o" "C:\Users\Dell\AppData\Local\Temp\cargo-installbxk00K\release\build\zstd-sys-5f68ba3594c32673\out\zstd/lib/common\debug.o" "-c" "zstd/lib/common\debug.c"
running: "gcc.exe" "-O3" "-ffunction-sections" "-fdata-sections" "-m64" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-DZSTD_DISABLE_ASM=" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-o" "C:\Users\Dell\AppData\Local\Temp\cargo-installbxk00K\release\build\zstd-sys-5f68ba3594c32673\out\zstd/lib/common\entropy_common.o" "-c" "zstd/lib/common\entropy_common.c"
--- stderr
error occurred: Failed to find tool. Is gcc.exe installed? (see https://github.com/alexcrichton/cc-rs#compile-time-requirements for help)
error: failed to compile anchor-cli v0.25.0 (https://github.com/coral-xyz/anchor?tag=v0.25.0#6baed77e), intermediate artifacts can be found at C:\Users\Dell\AppData\Local\Temp\cargo-installbxk00K
Error: Failed to install 0.25.0, is it a valid version?

Reference: https://www.anchor-lang.com/docs/installation
Did you install using Anchor version manager (avm) (recommended)?
I recommended installing through Build from source for other operating systems without avm

I have fixed this problem by switching my rust toolchain and installing VS studio packages


README: fix link to remoteobjects library

Coverage remained the same at 50.212% when pulling c26599f549df9710e6149d32a10d514d4e5c0371 on jcristau:readme-fixes into 20ea366aafcb0f68d2b98b9acd97b8c63b2efbd4 on mozilla:master.


[Bug]: RBAC - ME - Production given Execute only, Staging full permissions - manage env screen is not working correctly

Is there an existing issue for this?

[X] I have searched the existing issues

Description
When Production is given Execute only access, Staging given full permissions [Execute, Edit and View]- in manage env screen  we are able to select Production from the Configure environment dropdown, and also add host and port number and bind values.
Only Staging should be available in the Configure Env Dropdown
Steps To Reproduce

Go to Roles, create a new role say role5, give your workspace full permissions, and give ur DS {say firestore}, full permissions
For staging, give all the permissions (Edit/Execute and View) but for prod give only execute permissions
Save and add this role to a new group say group5 and add a new user (user5) to this group.
Signup and login as user5, click on the FireStore db from the left panel, go to edit Datasource page. Here on toggling between Prod and Staging, Prod bound values are masked while Stagings are shown - which is correct.
Navigate to manage environments page from here.
Here, the Configure env dropdown has Staging and Prod. Also, when Prod is selected, we can add host and port and bind values, we can delete the existing configuration fields and bind values.

Only Staging should visible in the Configure environment dropdown - Prod should not be available in the dropdown at all as it does not have view rights.
Also, there should be a way to handle the bottom toggle in this case.
Possible soln - For the manage env screen, the bottom toggle should default to Staging and Production should not be available as an option

Public Sample App
No response
Issue video log
No response
Version
EC2 instance

could you do a POC from client side on this?

I'll add the disabled tag to the host address and port tuple. Dropdown might be a bigger task to handle and would take much effort. I'll try to see if it is possible.


Fatal Error 400

!!! FATAL ERROR: Response status code does not indicate success: 400 (Bad Request).
System.Net.Http.HttpRequestException: Response status code does not indicate success: 400 (Bad Request).
at Spludlow.MameAO.ArchiveOrgAuth.GetCookie()
at Spludlow.MameAO.MameAOProcessor.Initialize()
at Spludlow.MameAO.MameAOProcessor.Run()

Hello @Pep0950
This is down to archive.org still not accepting logins after they where attacked.
Hopefully they will get it working soon.
See here
https://github.com/sam-ludlow/mame-ao/issues/54
Cheers
Sam

Just wanted to be aware, thank you.

Closing see here https://github.com/sam-ludlow/mame-ao/issues/54


Supporting ActiveRecord Scopes

For a DRYer coding behaviour common database interaction can be bundled with scopes right in the models (i guess you know how it works in Rails).
With Java/ActiveJDBC i would solve that with static methods in my Model extension classes, but this does not allow method chaining (ok i guess ActiveJDBC only allows method chaining for record creation at the moment with the set() method).
Would that be a feature for ActiveJDBC or don't you see it for the framework?
If not how would you use/solve such a behaviour at the moment?
As an idea a new class Scope could extend class LazyList and augment it with methods which allow method chaining.

Can you  propose an example of such API ?

Some sort of class variable/method with a type or annotation value 'scope' which allows to be called as you would call a static method and which contains database retrieval code against the table that model represents, e.g.
class Article < Model
  public static Scope published, -> { where(published: true) }
  // or
  @Scope
  public static published, -> { where(published: true) }
end

The following aspects would be important to consider:

allowed to combine (chain) several scopes (works as AND conditions)
-> this would mean that a scope returns the model class and not the results directly, as you do something similar with Model method set()
should be able to accept arguments, e.g.

Scope created_before, ->(time) { where("created_at < ?", time) }

I can see Ruby API:). Please, propose Java API.

This would mean that i'd have to spend more time with the framework for being able to propose the API, which unfortunately i don't have.
Meanwhile i will solve it with static methods and without chaining possibility.

I'm not asking for the implementation, but rather what the Java methods will look like in use.

Well in the single scope usage:
List<Articles> articles = Article.published;

and in the chained manner:
List<Articles> articles = Article.published.most_recent.ordered('asc');

is there a way to further specify selections with where?
Here:
List<Articles> articles = Article.published().most_recent().where("name = ?", "John").ordered('asc');

I guess the interesting part will be solving the problem that the return value of each scope should be Article.class (in our example). Then it should also be possible to mix it with e.g. a where clause.

It is a builder pattern, does not have to be a, Article class, could be just a new class called Scope or ScopeBuilder. However, Java has no method_missing, so no matter how you want to slice and dice it, the use of this API will not be as elegant as in Ruby:
public class Article extends Model{
    class ArticleScope extends Scope{
    public static ArticleScope most_recent(){
        super.setCondition("created_at > (now() - 30 days)")  ;
    }
    public static ArticleScope published(){
         return new ArticleScope("published = 1");
   }
}

So, in the end, you will have to write some ugly code regardless... :(

I'm also thinking back and forth. Without the chaining part i guess it's quite easy (lambdas).
Your different static methods could all collect the conditions where an additional extra call (e.g. performQuery()) at the end of the chaining then could a) put the conditions into on ActiveJDBC where clause and then perform the db query.

sure, can you provide an example  with lambdas?

public interface Scope {
    public List<?> performQuery();
}

public class Article extends Model {
    public static Scope most_recent = () -> Article.where("created_at > ?", "2018-03-14 12:49:00");
}

List<Article> articles = (List<Article>)Article.most_recent.performQuery();

ok, but how  would you chain multiple scopes?

That's what i meant: Without chaining it's easy, and with chaining it gets tricky.

right, but if you do not chain, there is not much value, right?

Depends on what the scopes do and how often you can use them, but yes: chaining would be a big plus here.

Please don't close that feature request. When i have time i will think about the chaining aspect again.

no problem

Ok i have the following proposal regarding (chaining of) scopes.
The Model class needs to be slightly extended by
public static IScopeBuilder scopeBuilder;

and
/**
     * Starts a new scoping action, therefore we have to set the model class.
     * @param scope
     * @return
     */
    public static IScopeBuilder add(Scope scope) {

        if (scopeBuilder == null) {
            scopeBuilder = new ScopeBuilder();
        }
        scopeBuilder.setModelClass(modelClass());
        return scopeBuilder.add(scope);
    }

The other files are attached.
A scope is defined like this in a Model extending class (very simple example):
public static Scope subtitles = new Scope("subtitle IS NOT NULL");
public static Scope titles = new Scope("title IS NOT NULL");

And then chaining of these scopes looks like this:
List<Show> list = (List<Show>)Show.add(Show.subtitles).add(Show.titles).performWhereQuery();

This is just a prototype. If elaborated i would say that e.g. OR scopes are also possible.
What do you think?
Archive.zip

How about this approach:
static class Article extends Model{
        static {
            scope("published", "published = 1");
            scope("unrated", "rated = 0");
        }
 }

...
List<Article> articles = Article.scopes("published", "unrated").where("... additional stuff");

This also looks promising.
Do you already know how scope and scopes could be defined/handled?

I think is should follow the same philosophy as validator framework, hence the static block: http://javalite.io/validations

Btw. passing in arguments would be a nice additional feature for scopes (see how it looks like for ActiveRecord).

haha, not so fast,  that was just a proposal of the API. I will take a look at it next week, time permitting...

Sure, no problem.

@rivella50 check this out. See usage here:
https://github.com/javalite/activejdbc/blob/c95b39c3a9281f9106f1ad43a3e6ad1648f96633/activejdbc/src/test/java/org/javalite/activejdbc/ScopeSpec.java

@rivella50  are you referring to this:
class Article < ApplicationRecord
  scope :created_before, ->(time) { where("created_at < ?", time) }
end

Article.created_before(Time.zone.now)

To be honest, I'm not too crazy about this feature. This will complicate the framework with a little benefit to the end user.
Have you tried the  new implementation ?

I can test it this evening within my current project.
But what i've seen so far looks great!

I've taken your ScopeSpec and tested in a similar manner.
Works all fine and gives the correct and filtered results which i compare with direct SQL queries against the database.

@rivella50 awesome, I will close it and write a  new documentation page on the site.

That's great news, thank you very much, Igor!

enjoy, here is the doc page: http://javalite.io/scopes

Cool, now i can start refactoring my current project.

@rivella50 haha, this is never a bad idea!


⚠ Cioos National CKAN Sitemap has degraded performance

In b01f7ab, Cioos National CKAN Sitemap (https://catalogue.cioos.ca/sitemap/sitemap-1.xml) experienced degraded performance:

HTTP code: 200
Response time: 6637 ms

Resolved: Cioos National CKAN Sitemap performance has improved in 0efa530 after 26 minutes.


Adding source_url to a new post

Checking the tumblr API I don´t think there is not a request parameter to add a source_url for the content.
For example creating a photo. The source parameter refers to the source for the picture, but the source_url or "content source" is actually the original source for the content.  Which is explained here:
https://support.tumblr.com/post/125279368827/heres-a-nice-thing-no-matter-how-many-times-an
It will be nice to be able to add that directly from the API when creating a new post.

Though not well documented, the HTTP API does accept a source_url string parameter for this, but (as you know) the python library does not. I think getting this working would only require adding 'source_url' to the list of post parameters at:
https://github.com/tumblr/pytumblr/blob/afe2c4c67068095771a1d3e70eba20fa4e513c08/pytumblr/__init__.py#L511
Are you interested in writing a PR?


1.2: Adjusted testcases to new error format of zhmcclient 1.3.3

Rolls back PR #288 into 1.2.1

Coverage remained the same at 32.582% when pulling c94b4d4776c0a2d2bae2490932feeb0cfff35070 on 1.2/andy/fix-error-test into 09451fa65354f58f91aefdeb00ee236b652bde0e on stable_1.2.


[BUG] SMTP not working gmail.

Description
When using gmail smtp with PHPMailer cannot send mail when registering new users or restarting password.
Steps to reproduce this issue

Provide MAIL configuration according to documentation and use gmail smtp e.g.

MAIL_USE_SMTP: "true"
MAIL_SMTP_HOST: "smtp.gmail.com"
MAIL_SMTP_PORT: "587"
MAIL_USE_SMTP_AUTH: "true"
MAIL_SMTP_USER<EMAIL_ADDRESS>MAIL_SMTP_PASSWORD: "XXX" <- this is app password from gmail
MAIL_FROM_ADDRESS<EMAIL_ADDRESS>MAIL_FROM_NAME: "blueprintUE self-hosted edition"
MAIL_CONTACT_TO<EMAIL_ADDRESS>
Run docker container
Try to register new user

Expected behavior
This should send registration mail and proceed to next page.
Actual behavior
Getting error: Error, could not create account (#500)
Screenshots

Additional context
all 'xxx' are actual correct gmail name for my company
I've fixed it by adding:
$this->mailer->SMTPSecure = PHPMailer::ENCRYPTION_STARTTLS;
in https://github.com/blueprintue/blueprintue-self-hosted-edition/blob/main/app/helpers/MailerHelper.php after line 34
I've fixed it for gmail smtp but not sure how it would work with local smtp.

Thank you for your issue.
I have added an option in the environment variables to enable or disable TLS: MAIL_USE_SMTP_TLS


How to use LLM deployed locally？

Your current code uses OpenAI’s API key to access the LLM service by default. I’d like to switch it to use a local LLM, which I have deployed through LLaMA-Factory and is accessible via a local API, for example, at http://localhost:7788/v1/. Could you guide me on how to make this adjustment? Thank you!

Upvote & Fund

We're using Polar.sh so you can upvote and help fund this issue.
We receive the funding once the issue is completed & confirmed by you.
Thank you in advance for helping prioritize & fund our backlog.

@deeper-coder we need function calling, if you can get a function calling model to work reliably then it will work. But you need a class with a run(task: str) or __call__(task: str) method to integrate into the ToTAgent class

I plan to use llama3 70B and I noticed that in the OpenAIFunctionCaller class, you’ve implemented the run method as shown in the image. So, can I achieve my desired functionality by passing base_url = "http://localhost:7788/v1/" in **kwargs?

@deeper-coder we need function calling, if you can get a function calling model to work reliably then it will work. But you need a class with a run(task: str) or __call__(task: str) method to integrate into the ToTAgent class


Migrate to Docker Swarm Services

https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/

https://geek-cookbook.funkypenguin.co.nz/docker-swarm/

$ sudo docker stack deploy -c docker-stack.yml monitoring
Ignoring unsupported options: devices, privileged, security_opt, userns_mode

docker swarm list dependencies of a service
docker stack ignoring unsupported options


dragging and copying causes crashes

Description
When I drag a sql from editor, it causes the application to crash
DBeaver Version
24.0.5
Operating System
win11
Database and driver
NA
Steps to reproduce

open sql editor
dragging a certain string

Additional context
Installed jdk11

We would kindly ask you to reproduce the issue once again. After you reproduce the issue, please send us the DBeaver log files.
The log files are generated automatically and called .log and .dbeaver-debug.log exactly. They are located in the %APPDATA%\DBeaverData\workspace6.metadata directory. Here is our wiki page on this matter: dbeaver.com/docs/wiki/Log-files/#log-files.
If there are any java fatal logs (https://dbeaver.com/docs/dbeaver/Log-files/#java-fatal-logs), please provide them too.

dbeaver-debug.log
.log

How did you install DBeaver: with installer downloaded from the website, from Microsoft Store, or do you use portable version?

Downloaded 24.0.4 from the official website, upgrading to the latest version still works like this.

@WarsenLiu Installed jdk11
DBeaver minimum JDK is 17. Could you try launching it using the JDK provided by the client if this is not the case now?

@Destrolaric I didn't specify the use of jdk11 in the ini configuration, I only configured it in the system env.

Please clarify what do you mean by "drag a sql from editor"

And what do you mean by "crashes"? I don't see anything fatal in logs. How does this "crash" look like?

There are too many situations that cause crashes. Last time, I only gave one example, and the following are the video and logs of dragging SQL to crash.
https://github.com/dbeaver/dbeaver/assets/77057177/4979fd61-6897-4c54-9893-689313f3d22d
dbeaver-debug.log
.log

Thank you for providing more information

I couldn't find it in the workspace, but every time I crash, it's generated in the exe directory.
hs_err_pid45168.log
hs_err_pid20276.log

I don't know if it's possible, but can you try turning off the Chinese input method popup to see if the issue still reproduces?

Even if the Chinese input method is turned off, it still crashes.
It seems to have nothing to do with this. My friend's Lenovo computer on win11 also experiences this situation.

For now we haven't managed to reproduce the issue.

Hello, Can you please check same case on another PC, if it is possiable? Seems native call lead to swt error.

It's been a while since no update here. If the issue is still actual please let me know and provide additional information.


Hunting queries windows event batch7

Change(s):
Add Windows Event Data to the following Hunting Queries:
Hunting   Queries/SecurityEvent/RareProcbyServiceAccount.yaml
Hunting Queries/SecurityEvent/RareProcess_forWinHost.yaml
Hunting Queries/SecurityEvent/RareProcessWithCmdLine.yaml
Hunting Queries/SecurityEvent/CustomUserList_FailedLogons.yaml
Reason for Change(s):
Part of adding Windows Event data source project
Version Updated:
No
Testing Completed:
Yes
Checked that the validations are passing and have addressed any issues that are present:
Yes

@aprakash13 Please review Hunting Queries. Thanks.

@oshezaf  Please review Hunting Queries. Thanks!!!

Hi @oshezaf
Can we close this and reopen whenever required as you have labelled it as differed.
Thanks

Done


Install script fixes

This PR fixes some of the errors I've got when trying to run oc+istio with the provided hack scripts.

@cfcosta I don't recall having issues with the istio scripts, so I am wondering - if this is going to be merged, will I now experience errors? :)

@abonas it was not working for me at all, can you check this branch to see if it does work?


Remove Bouncycastle as the default provider fixes #52

Previously, the library placed an artificial constraint needing
the BouncyCastle provider to perform cryptographic functions
when it was not necessary to do so.
This commit removes the requirement to both require BouncyCastle
as the sole provider, as well as requiring the BouncyCastle
dependency at runtime.
If the user wishes to use BouncyCastle, they should add the provider
as is done in the tests. The library will automatically use
whatever the preferred provider is instead of BouncyCastle.

:+1:

@brakthehack it might be good to document what functionality in jwt relies on BouncyCastle. Is there a subset of functionality that doesn't rely on any additional security providers?

Just to be sure, the migration path for users would be to add BouncyCastle as a dependency and then do:
Security.addProvider(new BouncyCastleProvider())

Right?

You've got it.

@brakthehack it might be good to document what functionality in jwt relies on BouncyCastle. Is there a subset of functionality that doesn't rely on any additional security providers?

So, I'm not all that familiar with the code base so I'm not sure if I can totally confirm, but my take is that this change affects the algorithms used, which is specified by the user. For example, if the user's token uses RSA signatures, it will probably work out of the box. However, if we remove BC as the provider in the tests we see a couple of tests in the core project fail:
[info] JwtSpec:
[info] pdi.jwt.JwtSpec *** ABORTED ***
[info]   java.security.NoSuchAlgorithmException: ECDSA KeyPairGenerator not available
...
[info]   transcodeSignatureToConcat and transcodeSignatureToDER
[info]   - should be symmetric
[info]   - should be symmetric for generated tokens *** FAILED ***
[info]     java.security.NoSuchAlgorithmException: ECDSA KeyPairGenerator not available

The primary reason is NoSuchAlgorithmException because it appears the default provider on my system does not provide a generator for ECDSA algorithms.
So to answer your question, it's system-specific which is why we probably should shy away declaring what is and is not supported.


Feature requests & bugfixes: bl-dataset-bidsupload.js

Feature requests:

handle sessions (longitudinal data) - need another level of nested dirs, e.g.: sub-subject/ses-session/anat/sub-sub_ses-session_t1w.nii.gz
allow links (currently, source data has to be a file not a link) - this might require a feature request or PR to node-archiver to add the -h (--dereference) option to tar

Bugfixes:

line 67 should read from participants.tsv
line 69 should look for "participant_id" column

@kathrynalpert thanks! @stevengeeky

I've added support for sessions, and fixed those bugs. I haven't updated to allow links, however.
@kathrynalpert What files are linked exactly? The directories, or the files? Or maybe both? Are they symlinks or hardlinks?

[Sorry for my delayed reply - I wasn't receiving notifications, but should be now.]
@soichih In my case, the links are symlinks to files, and it seems that your new code (1.2.38) works with them!

One comment about session handling: The upload works beautifully and the main "datasets" tab has the sessions listed nicely. However, when I'm staging datasets for processing, only the subject ID is visible (see screenshot, I'm talking about how to distinguish between last two datasets), so I have to use timestamp to determine which session I'm selecting. It'd be great if the session were visible here, too.

@kathrynalpert Thanks for the feature request. I've added session labels on the dataset selecter and a few other places. Please let me know if you notice anywhere else that I should display it. By the way, you can use this repo for posting Brainlife UI issue / requests > https://github.com/brain-life/warehouse/issues

Fabulous changes! I'm closing this issue and will make UI requests there in future.


keep specified imports without bundled

Is your feature request related to a problem?
Here's the problem:
index.html*
<script type="module" src="./a.ts"></script>
<script type="module" src="./b.ts"></script>

and both inside a.ts and b.ts there's imports "./c.ts".
packup build results the js code of c.ts included both in a.js and b.js.
Describe the solution you'd like
intead of bundle everything imports,
could packup supports // !packup=rules comment like following?
// !bundle=module
import "./c.ts";

"// !bundle=module" bundle c.ts as c-###.js and change import to import "./c-###.js"; in the final result.
// !bundle=off
imports "https://demo.com/c.js";

"// !bundle=off" keeps the followed imports without changed.

btw, could the output of js files keep the name as its name prefix?
it seems named by the html file name right now - if more than one html using the same m.ts, there will be more than one copy with different names.

here comes my fork without wasm and update deps' versions https://github.com/mindon/packup
// usage
// !bundle=off|module or customized directory (start with / as dist root) or name (without suffix /)
// ;

// !bundle=/shared/depends
import { something } from '../deps.ts';

this example generate depends.md5-code.js under dist/shared/, and keeps using
import { something } from './shared/depends.md5-code.js';

@mindon Great!


Add HD-Wallets

This PR includes BIP 32, 39, 44. A GUI for setting up mnemonic/hd wallet using mnemonic. Also A RPC and GUI to upgrade non-HD wallets to HD wallets. Using HD wallet you will not be able to cold stake until we implement generating Cold staking addresses as Pivxes implementation makes these separate addresses.
Setup Mnemonic page
First page
This says Phore, but I already fixed that

Restore Page
You can restore your wallet with mnemonic words

New Wallet Page

Sort Page
This page is to make sure they have wrote down their words, it is also so they remember the order as the order of the words is very important

Main Wallet Screen
HD Status
I have also added a HD status button, normally I wouldn't bring this up, but yeah just cause I had to add it for the new ui

Wallet Repair
Her I added a button to upgrade non-HD Wallets to HD. This would also use the wallet startup page just kind of reusing that.

RPC
There are a few changes because of the HD addition, but I also added a new rpc command to upgrade wallets from the client also.

will test shortly,and possibly get travis or something going to build it

utACK, compiling and testing, will report back after testing all features and reviewing code.

A button seems to be invisible here, not sure if this PR introduced the bug.
Although in ZENZO Core v1.3.0 your last CSS changes broke some buttons that needed fixing after merge, perhaps it's the same issue?

A button seems to be invisible here, not sure if this PR introduced the bug.
Although in ZENZO Core v1.3.0 your last CSS changes broke some buttons that needed fixing after merge, perhaps it's the same issue?

That issue was made by @akshaynexus when he rebased

Will take a look soonish if that was the cause for missing buttons


Feature/Twitter scraping w/o creds

Allow Nodes to participate in twitter scraping w/o bringing their own creds
Clean up obsolete pg integration
Added get bootnodes from deployment json on s3

Hey @jdutchak just trying to get some context here on the not logged in scraping of twitter? Whats the thinking here? Help me catch up when I get back into the seat on Thu/Fri

Love that we are getting this out @jdutchak

Assumptions
-- what node can satisfy work?

scrape tweets
scrape webpages
scrape discord, reddit, in progress
use case node A makes a request to a node(n) that is staked and opts in to participate for scraping


Update identity_management_FAQ (language edit)

I've edited the answers for clarity and grammar while maintaining the same structure and whenever possible the same verbiage.
Description
Mainly this involves smoothing out some rough edges. Changing the order of certain things, removing duplicated words and phrases so things flow better.
Motivation and Context
A FAQ is often the first place someone will go when they're not intimately familiar with the product or the issues at hand. It's therefore particularly important for it to be written as clearly and concisely as possible.
How Has This Been Tested?
N/A
Checklist:

[x] I have added a Signed-off-by
[x] Either no new documentation is required by this change, OR I added new documentation
[x] Either no new tests are required by this change, OR I added new tests
[x] I have run goimports, go vet, and golint. I have cleaned up all valid errors and warnings in code I have added or modified. These tools may generate false positives.  The goal is clean, consistent, and readable code.

Signed-off-by: Joe Alewine<EMAIL_ADDRESS>The changes I believe help the readability and therefore the usability of the FAQ.

I assume this patch still has outstanding work before merge consideration?

ghaskins --
I've asked Josh Horton to contribute to this matter so I can make the appropriate edits. As of right now some issues need to be resolved before this can be merged.

@joe-alewine thank you for the update

Who wrote the original FAQ - my instincts say we should at least try to engage that person(s) ... and also to compare what this "FAQ" says to the protocol spec content in this area. ... (As an aside, Joe, we will want to split up protocol spec into separate documents.) ... (i.e. would this "FAQ" content do better for the user in a clear and integrated and separate "membership services" document. Then we can get rid of the question of identifying a "unique" feature, and simply document what is.)

So the plan is to do away with a centralized FAQ and establish separate FAQs for each topic? Or would the kinds of specific process questions that make up the FAQ right now be folded into documents related to the various topics while a new, broader centralized FAQ would be created from scratch and take its place?

LGTM thanks @joe-alewine

Thanks, LGTM


Using sync storage

What is your opinion on using chrome's sync storage to persist config settings to the cloud so that they can be shared between installs linked to account?
docu: https://developer.chrome.com/extensions/storage

+1

+1, love this as you can pop a new install of chromium and get your setup working instantly!
Would it be used for the account connection as well?

Account connection wouldn't be a wise thing IMO but syncing everything else would be a good move!

@david-sabata what is your stance on this? Is this something that you think should be included? Or should we leave the storage to local only?

I poked around this a little when I was switching storage from localStorage to "extension storage" (it's probably not a real term, but it is somehow different from classic localStorage which was used before).
From what I remember there were some limitations, but probably nothing critical. We don't store much data yet. Also I'm not sure about the security, but it would be super nice if we could sync the tokens too. Because besides that there isn't much configuration that needs to be stored. Currently we store tokens to some internal storage and I'd say that the cloud storage would work and be secured quite the same.
It would be cool to sync also manually entered track info some day, but we may hit the limit with that.

Also I'm not sure about the security, but it would be super nice if we could sync the tokens too.

Unfortunately:

Confidential user information should not be stored! The storage area isn't encrypted.

Currently we have the following storages:

Connectors for connectors options;
customPatterns for user-defined URL patterns;
Options for the extension options;
LocalCache for edited song metadata;
Core for core data;
storages for each scrobbling service (Last.fm and Libre.fm).

Althought all these local storages can be quite easily transformed into sync ones, there're some reasons to keep some of them as local ones.
I suggest to make first three storages as sync.
LocalCache storage is supposed to have a lot of data, while sync storages have some limits.
Core storage contains app version and notification flags. Propably, it makes a sense to sync these flags; to achieve this we can split Core storage into several ones.
Last.fm and Libre.fm storages should be stored locally.

@alexesprit what does core contain? by notification flags do you mean whether to display the notifications?
I agree LocaleCache  could become problematic. We could plan some larger storage migration strategy such as export/import or even cloud handling in future iterations.

@inverse core storage contains the extension version (not used now, but could be used for specific migration process) and notification flags (don't know how to call them properly). These flags used to prevent displaying notable changes several times:
https://github.com/david-sabata/web-scrobbler/blob/cf0d8b8455ef1ca45043190e2890f9d106a2c1b9/core/background/main.js#L330-L350
I updated this code a little (reverted true and false meaning), but the idea is still the same.

Nice - I haven't seen this code for a while - It looks a lot nicer with the new ES touches :)
That makes sense.


Retain copyright notice in distribution files

The copyright notices seem to be removed from the dist/ files after building. We should retain them.
JavaScript bundling tools will usually leave in comments that start with /*! so that license headers can remain intact.

at worst we can add it to the top of each file after the build.


Update parameter types of RegExp compile method

This PR updates the parameter types of RegExp compile method. The parameter names and types follow the convention used in RegExp constructor (line 952).
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp/compile
Fixes #46030

/cc @orta if you want to include this in any roundup of lib changes for the docs

Maybe it could go in breaking changes, but it's a deprecated function which probably might not have even worked before (given that you need that first param to do anything realistically) so it's probably OK to just fly under the radar


rangeproof_getheader_impl return values are changed but call sites are not updated

It looks like the only difference between the upstream secp-zkp and the vendored version in this library is that rangeproof_getheader_impl returns a more detailed error return code. However,

This is called from rangeproof_verify, which will now fail 100% of the time.
The new return value is propagated up from rangeproof_info, which is in the public API, but again the code is basically converted to the old one by the Rust bindings.

Was this an intentional change? If so, it may make sense to propose it upstream, or at the very least include it as an explicit .patch file which is used by the vendoring script.

Oh yes, this is an unintended left-over where we were debugging our bindings!
Those return values should be reset to the original ones!


To make its footer more effective

I want to do some changes in the footer to make him more attractive by changing the background and text fonts colors etc.
so assign me this task.

Go ahead @Kirankp25.
Assigned to you.

@Kirankp25 any updates?

yes, today I will make PR for it


Dumpulator takes a long time to load when handling large dumps

Feature Request: Please look into addressing the issue where large dump files aren't easily processed by dumpulator.
Description: Dumpulator is taking an excessively long time to load a dump file of 40MB. I was testing the decryption of an RC4 encrypted configuration block from gh0stRAT. I dumped out the process and it was 40MB. I fed this dump into dumpulator and it was still trying to load after 20 minutes.
Sample: https://www.virustotal.com/gui/file/0a9c881b857d4044cb6dfeba682190d7d9dc6ef94bc149cac77f3e0f65e9c69a

Yeah, this is something I've been looking into. Locally I have some changes related to it, but it requires a bit of a rewrite. The main issue (which is becoming a bit of a recurring theme) is that unicorn has a memory allocation strategy that's O(n). This means that you get effectively O(n^2) when allocating all the ranges. I did a test and allocating a 1000 ranges takes 600 seconds or something ridiculous like that.
The solution is to keep our own memory map (which is already done to a great extent) and lazily map the pages into unicorn when we get a fetch exception. This can definitely be done, but it requires some work to do cleanly and some heuristics to not tank performance completely (you want to map whole regions for example, not single pages).

May I ask when will this problem be solved? I loaded a 80MB file, and it took nearly an hour to finish, so I had to give up.

Whenever I have time. Currently I’m on vacation.

@c4sku7 @gakar06 could you try again with https://github.com/mrexodia/dumpulator/pull/36 this issue should be largely mitigated. I tested on a 4GB dump and it takes around 25 seconds to load. Not perfect, but previously it took something like 45 minutes...


Uncaught (in promise) DOMException: The element has no supported sources.

Uncaught (in promise) DOMException: The element has no supported sources.
I dk why do i get this, is it linked with React Native ?
There are so many of these,

doing on ios.
Please read the following carefully before opening a new issue.
Your issue may be closed if it does not provide the information required by this template.
We use GitHub Issues exclusively for tracking bugs in React Native.

If you have a question, ask on Stack Overflow: http://stackoverflow.com/questions/tagged/react-native
If you have a feature request, post it on Canny: https://react-native.canny.io/feature-requests

Make sure your issue reproduces on master. Your issue may already have been fixed!
If your issue is present in master and in the stable release, try to reproduce your bug on https://snack.expo.io/
If you can't reproduce the bug on Snack, provide a sample project.
At the very least, provide an example of your code.
--- Delete everything above this line ---
Description
Explain what you did, what you expected to happen, and what actually happens.
Reproduction Steps and Sample Code
List all the steps required to reproduce the issue you're reporting. These steps should be clear and concise.
An example of your code is REQUIRED
Solution
What needs to be done to address this issue? Ideally, provide a pull request with a fix.
Additional Information

React Native version: [FILL THIS OUT: Be specific, filling out "latest" here is not enough.]
Platform: [FILL THIS OUT: iOS, Android, or both?]
Development Operating System: [FILL THIS OUT: Are you developing on MacOS, Linux, or Windows?]
Dev tools: [FILL THIS OUT: Xcode or Android Studio version, iOS or Android SDK version, if applicable]

Hi there! This issue is being closed because it has been inactive for a while.
But don't worry, it will live on with Canny! Check out its new home: https://react-native.canny.io/feature-requests/p/uncaught-in-promise-domexception-the-element-has-no-supported-sources