Refactor server startup APIs and Https support

The previous support for SSL was not great - you were locked into
OpenSSL and had no ability to configure it the way you wanted. Hyper is
going to be cutting a release soon that completely removes the built-in
OpenSSL and Secure Transport functionalty in favor of external crates.
While we're at it, also allow a NetworkListener to be manually
created, and refactor the configuration a bit to avoid combinatoric
blowup in methods.
Closes #483
Closes #447
Closes #424
r? @untitaker
It probably makes sense to land this along with the hyper upgrade. @seanmonstar, do you have an idea on when that'll be roughly?
The optional OpenSSL dependency hopefully won't cause problems, but it seemed important to keep the https example around.

Oops, wrong @seanmonstar..

I waited on that issue for a few days to allow discussion, it seems it's wrapped up. I'm away from the computer today, but I can probably finish the release tomorrow.

Ok, I've switched the https example over to native-tls, which should make things nicer for OSX and especially Windows users.

Can we re-add the tests for this in Travis?

Done

Thanks!


Unhandled exception in script

Environment
OS: Windows 11 (26100.2454)
Python: 3.10.7
CUDA Version: 12.1
ffmpeg version on path: 6.1.1
Downloaded and installed latest build from here and getting the following error.
Failed to execute script 'main' due to unhandled exception:
tokenizers>=0.20,<0.21 is required for a normal functioning of
this module, but found tokenizers == 0.15.2.
Try: `pip install transformers -U' or `pip install -e '.[dev]" if you're
working with git main

Traceback (most recent call last):
  File "main.py", line 4, in <module>
  File "buzz\buzz.py", line 58, in main
  File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
  File "buzz\cli.py", line 22, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
  File "buzz\widgets\application.py", line 16, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
  File "buzz\widgets\main_window.py", line 23, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
  File "buzz\file_transcriber_queue_worker.py", line 16, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
  File "buzz\transcriber\whisper_file_transcriber.py", line 21, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
  File "buzz\transformers_whisper.py", line 7, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
  File "transformers\__init__.py", line 26, in <module>
    from . import dependency_versions_check
  File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
  File "transformers\dependency_versions_check.py", line 57, in <module>
    require_version_core(deps[pkg])
  File "transformers\utils\versions.py", line 117, in require_version_core
    return require_version(requirement, hint)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "transformers\utils\versions.py", line 111, in require_version
    _compare_versions(op, got_ver, want_ver, requirement, pkg, hint)
  File "transformers\utils\versions.py", line 44, in _compare_versions
    raise ImportError(
ImportError: tokenizers>=0.20,<0.21 is required for a normal functioning of this module, but found tokenizers==0.15.2.
Try: pip install transformers -U or pip install -e '.[dev]' if you're working with git main

@mriamnobody Short term fix is to uninstall old version before installing the new version of the Buzz.

This is fixed in the latest development version https://chidiwilliams.github.io/buzz/docs/faq#9-where-can-i-get-latest-development-version


Remove horrific things from ast.hpp

I think that this is a significant improvement to the project.

Should be
namespace amongus = ::std;


Unable to save results as CSV or JSON. EXPORT, Open Results and Save Results all broken.

After running a query, 'Save as CSV' or 'Save as JSON' options do not work.
Steps to reproduce the behavior:

Go to SQLTools and run a query.
Click on EXPORT, 'Open Results', or 'Save Results'
For EXPORT, nothing happens and no notification.
For Save Results/Open Results, See error

Expected behavior:
A saved CSV file or JSON file on hard drive. XLSX would be nice too, but out of scope for a fix.
Screenshot:

Desktop

SQLTools Version 0.23.0
VSCode Version: VSCode Version 1.58.0-insider
OS: Ubuntu 20.04.2 LTS focal
Driver:

PostgreSQL/Redshift

Additional context
It has never worked, including on older versions to my knowledge, for the last 6-12 months.

I am having precisely this issue with a different configuration:

SQLTools v0.23.0
VSCode v1.58.2 (user setup)
Version: 1.58.2 (user setup)
OS: Windows_NT x64 10.0.18363 (Windows 10 1909)
Driver: SQLTools Intersystems IRIS v1.1
Database: Cache for Windows (x86-64) 2018.1.5 (Build 659U)

The offending code appears to be here:
https://github.com/mtxr/vscode-sqltools/blob/ba294769be006e9e605d70ca487e5ce6fb4126c8/packages/plugins/connection-manager/language-server.ts#L56
What it looks like to me is that the cache is never set; any successfully executing response should invoke the function defined at https://github.com/mtxr/vscode-sqltools/blob/8ee200af8cd659eb868f2889f0c42dd9c889842a/packages/plugins/connection-manager/cache/handlers.ts#L41, but this doesn't appear to happen with the extensions.
@sdissegna-maystreet, would you please take a quick look at this issue? Thank you.


Add PackedIterableDataset, peft_module_casting_to_bf16 util method and append_concat_token flag

What does this PR do?

Fixes https://github.com/huggingface/trl/issues/746 by introducing PackedIterableDataset
Inline with QLoRA code when using bf16 here, added peft_module_casting_to_bf16.
Added append_concat_token flag so that datasets which already have eos_token at the end of each sample can skip appending the eos/concat token.

Details:
Issues with ConstantLengthDataset:

Learning rate schedulers like Cosine, linear_decay ... will run improperly because the length of the dataset is set to the initial dataset while an epoch will have much fewer steps because of packing.
When max_steps is passed which has more steps than the initial len(dataset), it results in https://github.com/huggingface/trl/issues/746.

PackedIterableDataset is meant to fix this issue as it doesn't contain the length attribute and works as expected when passing max_steps.

cc @younesbelkada, @lvwerra

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

Closing this in favour of https://github.com/huggingface/trl/pull/1110


How to create Gremlin queries?

I noticed that the .NET SDK has a  CreateGremlinQuery method, but the Python SDK doesn't. I am trying to run a simple query against a graph DB (e.g. g.V()) and it's failing error code SC1001 and message Syntax error, incorrect syntax near 'g'..
Here's an example of a call I am making:
collection_link='dbs/mydb/colls/mycoll'
db_query='g.V()'
client.QueryDocuments(collection_link, db_query))

Am I missing something or is this simply not implemented/included in the Python SDK yet? If latter, any plans/ETA on when this is going to be supported?

@peterj this SDK only provides documentdb api of cosmos db. There is no gremlin api here.

Is there another Python SDK I could use to make Gremlin against DocumentDB?

@jayantacs knows more about Gremlin support. @jayantacs  can you please respond.

Closing this issue due to age. Active development of our Python SDK has moved to https://github.com/Azure/azure-sdk-for-python. We have also released 4.0 which includes many updates and new features. If this issue still needs our attention, please reopen it in the new repository.


[WIP] Rooms

Added backend functionality to

create a room with parameters
join said room
rejoin same room if disconnected
get room info
relay user typing input to others in same room

Awesome


Is it working already?

Hi,
I'm in need of a m3u8 parser for the browser, and all available solutions need node or CommonJS. Are you solution ready to work?

It should work, however this only supports a subset of functionality. We initially planned to expand it, but soon swapped out the application we were using it for with JSON responses, so we didn't do so. If you're looking for a base to build on, this could work, but it's not a full-feature library.

I don't need the whole spec to be implemented, just the most important feature. Since my project is using Angular I'm "porting" it from CommonJS modules to Angular Factories.
I'll update here the AngularJS fork and if in a future I implement any new feature I send a PR.

Sounds good, looking forward to PRs :smile:


Remove support for building fuzzers from configure.

Requires https://github.com/google/oss-fuzz/pull/7467 to be merged in first.

Ready for review.

@Dead2 should I do another PR to remove sanitizers from configure script as well? It is not well supported and it doesn't do all the compiler flag checks that cmake does. Either way, let me know what you think.

@nmoinvaz I would be good with that, yes. 👍


About the test data in your paper

In your paper, you have testing datasets selected as a subset from ADE20K and pascal, such as ADE-Car-12 and ADE-car-5. How to obtain these datasets? Because an identical testing dataset is core to a fair comparison. Thank you.

Thank you for your interest! We can't release ADE-car testing dataset directly due to the dataset license.
A benchmark leaderboard will be released soon.


script error

I have set up the A record, but it prompts an error

I guess you have to install the package 'bind-tools' or something similar, as the script wants to use dig, which seems not to be installed on your machine.

As @scyllaL mentioned, it seems that it is related not being able to use dig.


[docs] Do Not Merge (demonstration purpose of PR 34588)

This PR is here to demonstrate how CI in #34588 behave

Demonstrated PR has been merged


Add MultiVM to VM tests

This PR aims to test the reachability between multiple VMs.
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[X] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

/test all

Will update!


google sign in web How to detach appClientId

<!-- release-->

I want to separate the firebase of the commercial environment and the development environment and apply it to one project. What can you do?
Metatags appear to read only one value. Can you divide the two prices?

@minjinDelable
Your question is better suited for support channels like StackOverflow, so please ask the same there.
Closing from here, as this doesn't seem to be a bug or a feature request for the package.


Using foldRight to implement map / filter etc in List is relatively inefficient

Better to use raw iteration - affects Seq and LazySeq.

Released in 10.0.0-M4


Add destination space and org to network-policies command

Does this PR modify CLI v6 or v7?
v6
What Need Does It Address?
In a previous PR we added the ability to add/remove cross space network policies: https://github.com/cloudfoundry/cli/pull/1504.
This PR provides parity for the network policies command. It will now show the cross space network policies that exist along with its destination space and destination org.
Who Is The Functionality For?
Users who add cross space network policies.
How Often Will This Functionality Be Used?
It is our most common feature request from users.
Possible Drawbacks
This PR and the previous PR sets a precedent for cross org/space actions.
Why Should This Be In Core?
The network-policies command already exists in the core cli.
Description of the Change
Design doc: https://docs.google.com/document/d/17FtmNkI5iJ5Dxjgg6ToIefJeCtAEyc6f1rAZv3d5p8I/edit?usp=sharing
How Urgent Is The Change?
The previous PR to {add/remove}-network-policies has already been merged, so this will bring network-policies to parity. And we want this to have parity before releasing a new cli.
Other Relevant Parties
@jsirlin @ameowlia @oozie @tylerschultz

@christianang Thanks for this PR! Could you rebase this so that it can be merged without conflicts?

@jenspinney Rebased! Thanks for letting us know!

Amelia says meow.

Made some updates based on the feedback. Let us know if there is any other areas that can be improved or not.

@tjvman I've updated the tests per your feedback. Thanks.


Wrong CPU resource unit

In chapter Resource reservations, an example of CPU resource reservation is described, but the unit is incorrect.

CPU - 60ms

Should be core or millicore.
Ref: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-cpu

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a445f8cc-6581-ad43-1dba-719a94eda81a
Version Independent ID: 31280746-d8de-91b4-d430-91693c2e869d
Content: Concepts - Kubernetes basics for Azure Kubernetes Services (AKS)
Content Source: articles/aks/concepts-clusters-workloads.md
Service: container-service
GitHub Login: @iainfoulds
Microsoft Alias: iainfou

When you assign your own resource requests and limits, yes, it should be defined in core or millicore. The sample YAML manifests correctly show this further down in the article.
The resource reservations discussed in this section aren't pod requests or limits, rather how much CPU time and memory the Azure node is reserving for its own management operations. These reservations aren't anything you define or can change. They are noted here as if you view resource utilization on the host, you need to account for the reserved CPU time and memory that is not allocatable for your own workloads.
#please-close

@iainfoulds Thank your for reply.
Maybe my hyperlink before will lead to ambiguity, I just want to show the representation of CPU resources in kubernetes. I know the Resource reservations is Reserved for kubernetes & system, and it's not pod requests or limits.
Here is the Reserved example, the CPU resource unit is core or millicore. So I'd like to confirm that whether azure uses a different representation to declare CPU resource reservations, and what's mean of 60ms?
Thank you!

60 millicores, clarify/fixed by https://github.com/MicrosoftDocs/azure-docs/commit/5bac92997c4d7eeeed8517724bff5d7af1897aed#diff-6de4f7f0fd5600da4144e29d0ff8c53f

CPU - dependent on cluster configuration and node type

CPU cores on host
1
2
4
8
16
32
64

Kubelet (millicores)
60
100
140
180
260
420
740

Memory - 20% of available memory, up to 4 GiB max


Update 404 to point to the correct repo page.

Potentially we could just remove the link as well, if that makes more sense.

Thanks!


FEATURE - Enable users to select from google's Criteria or URL ad adapters

@DylanBaker can you work on adding this variable?
Note - this may change depending on the strategy taken w/ this open issue: https://github.com/fivetran/dbt_google_ads/issues/7

This feature is no longer possible to add with the sunsetting of AdWords and the new GoogleAds API for our connector. As such, I will close this FR.


Connection is poisoned: Cannot read framed message: Cannot read data: early eof

Thank you for the great crate!
I got the following error.  Can I fix this error? I would appreciate it if you could share it with me if you have a good idea,
Errors:
Failed to connect to broker","e":"cannot sync versions: Request error: Connection is poisoned: Cannot read framed message: Cannot read data: early eof

rskafka version: 0.5.0
Apache Kafka version: 2.6.2
I use Amazon MSK and have three brokers.
source code:
use rskafka::{
    client::{ClientBuilder, SaslConfig},
};

let client =
    ClientBuilder::new(connection)
        .sasl_config(SaslConfig::Plain {
            username: sasl_username,
            password: sasl_password,
        });;

debug!("start building client");
let client = client.build().await?; // This line causes errors.

i am seeing this erros too...

mine is failing on this:
 let (record, high_water_mark) = stream
            .next()
            .await
            .expect("some records")
            .expect("no error");

I found that I need to use SCRAM-SHA-512 as sasl.mechanisms in my case, so if I want to use rskafka, I have to implement SCRAM-SHA-512 in SaslConfig probably. However I guess we may be able to improve rskafka to output more reasonable errors.

i am not even using sasl at all but still getting the error

Sounds like the broker is cutting the connection, probably due to some protocol error. The best thing to try is to capture the network traffic (e.g. w/ wireshark) and analyze it. My rough guess is that is a bit dated and it doesn't like the newer protocol versions that we use.


Ignore BaseURL

When you use sandbox and production services they have different baseURL. I added the possibility to ignore the baseURL when matching url requests. If this is useful i can create a PR from my fork.

Sounds cool. PR away


Scarlet Monastery problems

Hello. All Scarlet Monastery groups made are automatically listed under graveyard, with no groups ever showing up under cathedral, armory or library. This means that once you're out of level range for Graveyard, you can no longer use ClassicLFG to find ANY scarlet monastery groups.

For a temporary fix you can check "Always show all dungeons" in settings. I've submitted a fix for this issue in pull requests and hopefully it'll be integrated soon.

Fixed.


PsrAutoloadingFixer - introduction

As psr_0 and psr_4 fixers share a lot of code after discussion with @keradus we want to merge them and introduce this fixer to replace them.

@kubawerlos can rebase and squash this PR for me?
I was wondering about two things:

PSR0 en PSR4 have the same code samples, yet one has only, why?
with PSR0 and PSR4 deprecated, how to configure the new one to do either PSR0 or PSR4?

@SpacePossum rebased and squashed

PSR0 en PSR4 have the same code samples, yet one has only one, why is that?

PsrAutoloadingFixer has 2 code samples - 1st without configuration and 2nd with configuration and Psr4Fixer is not configurable fixer, so it cannot have the 2nd sample

with PSR0 and PSR4 deprecated, how to configure the new one to do either PSR0 or PSR4?

If you use PsrAutoloadingFixer without dir then it works as psr0 or psr4 without config - that was the reason for the merge - both work the same with default config. Adding dir makes the fixer works as psr0 with a dir.

thanks for explaining, I leave the review to someone with more knowledge on this topic

@keradus is this still something you would like to have merged?

@kubawerlos please rebase, after that RTM :+1:

@SpacePossum rebased

Thank you @kubawerlos.


nil deference in TestUpdateRemoteReadConfigs

What did you do?
trigger Test TestUpdateRemoteReadConfigs
What did you expect to see?
PASS
What did you see instead? Under which circumstances?
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xaecd95]

goroutine 65 [running]:
testing.tRunner.func1.2(0xbbf420, 0x116cce0)
	/usr/local/go/src/testing/testing.go:1143 +0x335
testing.tRunner.func1(0xc000341400)
	/usr/local/go/src/testing/testing.go:1146 +0x4c2
panic(0xbbf420, 0x116cce0)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
github.com/prometheus/prometheus/storage/remote.NewReadClient(0xc0003382e0, 0x6, 0xc0001b9cc0, 0x20, 0xc0001b9dd8, 0x0, 0x0)
	/repos/prometheus/storage/remote/client.go:125 +0x175
github.com/prometheus/prometheus/storage/remote.(*Storage).ApplyConfig(0xc00030cb40, 0xc0005ffe78, 0x0, 0x0)
	/repos/prometheus/storage/remote/storage.go:113 +0x3e8
github.com/prometheus/prometheus/storage/remote.TestUpdateRemoteReadConfigs(0xc000341400)
	/repos/prometheus/storage/remote/storage_test.go:88 +0x27c
testing.tRunner(0xc000341400, 0xcc03c0)
	/usr/local/go/src/testing/testing.go:1193 +0xef
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:1238 +0x2b5

Similar issue with #10033, missing URL in
func TestUpdateRemoteReadConfigs(t *testing.T) {
	dir := t.TempDir()

	s := NewStorage(nil, nil, nil, dir, defaultFlushDeadline, nil)

	conf := &config.Config{
		GlobalConfig: config.GlobalConfig{},
                 // URL is missing
	}
	require.NoError(t, s.ApplyConfig(conf))
	require.Equal(t, 0, len(s.queryables))

	conf.RemoteReadConfigs = []*config.RemoteReadConfig{
		&config.DefaultRemoteReadConfig,
	}

Environment

System information:
insert output of uname -srm here

Prometheus version:
insert output of prometheus --version here

Alertmanager version:
insert output of alertmanager --version here (if relevant to the issue)

Prometheus configuration file:

insert configuration here

Alertmanager configuration file:

insert configuration here (if relevant to the issue)

Logs:

insert Prometheus and Alertmanager logs relevant to the issue here

Fixed in https://github.com/prometheus/prometheus/pull/10935


Error/Succes displaying

[x] Failed login
[ ] Failed PIN edit
[ ] Success collections setup
[ ] Clarify that the PIN toggle checkbox requires setup collections
[ ] Document maintainer process
[ ] Load current view/list rule for viewing slogans into PIN toggle checkbox

This can use the same display mechanic as the signal receiving!

Failed login notification since f0efd5b51c5bf1b03d786d4f0ca8ab4ce77bdbfe !

Failed pin edit 9d205fe6b59e50e3ef517d8e8e868680e523221d

Successful collection setup (among others) since 2816dcabccc50078f3b3d274c8cb1167287af917

Load current view/list rule for viewing slogans into PIN toggle checkbox since 35c5a2fd61a18cc68523ef49ecefd6d9aaa0abdc

Clarify that the PIN toggle checkbox requires setup collections since 3bb9fe81cd6c782d833ddcef6d639870089b7e59, closing the issue


C#: Rename and restructure control flow graph entities

Follow a naming structure similar to the data flow library:

ControlFlowNode -> ControlFlow::Node.
CallableEntryNode -> ControlFlow::Nodes::EntryNode.
CallableExitNode -> ControlFlow::Nodes::ExitNode.
ControlFlowEdgeType -> ControlFlow::SuccessorType.
ControlFlowEdgeSuccessor -> ControlFlow::SuccessorTypes::NormalSuccessor.
ControlFlowEdgeConditional -> ControlFlow::SuccessorTypes::ConditionalSuccessor.
ControlFlowEdgeBoolean -> ControlFlow::SuccessorTypes::BooleanSuccessor.
ControlFlowEdgeNullness -> ControlFlow::SuccessorTypes::NullnessSuccessor.
ControlFlowEdgeMatching -> ControlFlow::SuccessorTypes::MatchingSuccessor.
ControlFlowEdgeEmptiness -> ControlFlow::SuccessorTypes::EmptinessSuccessor.
ControlFlowEdgeReturn -> ControlFlow::SuccessorTypes::ReturnSuccessor.
ControlFlowEdgeBreak -> ControlFlow::SuccessorTypes::BreakSuccessor.
ControlFlowEdgeContinue -> ControlFlow::SuccessorTypes::ContinueSuccessor.
ControlFlowEdgeGotoLabel -> ControlFlow::SuccessorTypes::GotoLabelSuccessor.
ControlFlowEdgeGotoCase -> ControlFlow::SuccessorTypes::GotoCaseSuccessor.
ControlFlowEdgeGotoDefault -> ControlFlow::SuccessorTypes::GotoDefaultSuccessor.
ControlFlowEdgeException -> ControlFlow::SuccessorTypes::ExceptionSuccessor.

The change itself looks very reasonable, however I question whether this needs to go into the RC branch. We should also create some deprecated aliases for the old symbols (particularly, ControlFlowNode) before removing them completely, and we should have a change note.

The change itself looks very reasonable, however I question whether this needs to go into the RC branch. We should also create some deprecated aliases for the old symbols (particularly, ControlFlowNode) before removing them completely, and we should have a change note.

I did add entities for backwards compatibility: https://github.com/Semmle/ql/blob/84da5a36090b01c8f24d96c5ab36c6cfb284041f/csharp/ql/src/semmle/code/csharp/controlflow/ControlFlowGraph.qll#L3155
Sure, I can add a change note.
I targeted the RC branch because I also removed all compiler warnings on this PR.

Sorry Tom, I hadn't seen the aliases when I searched for them due to the large diff being hidden.

There seems to be repetition in SuccessorTypes::BooleanSuccessor. Would it be any clearer to name them SuccessorTypes::Boolean instead?

There seems to be repetition in SuccessorTypes::BooleanSuccessor. Would it be any clearer to name them SuccessorTypes::Boolean etc instead?

I prefer the naming as is; the nested SuccessorTypes module is mostly there to limit the number of entities inside the ControlFlow module.


migrated from apollo-client-preset to apollo-boost

Since apollo-client-preset has been deprecated, I have migrated the with-apollo example deps and code to utilize apollo-boost.

:ok_hand:

@ads1018


EFO3 for 19.04: Review of changes compared to EFO2

Running the pipeline with EFO3 results in an increase of associations from 3 to 6 million. This change needs to be investigated so that necessary adjustments can be made before the 19.04 Open Targets release.

[ ] Identify diseases that are only in the "disease" therapeutic area, and no others [AF]
This list has been generated by Adam.
[ ] Analyse where the new associations come from - new terms or increase in associations for existing terms [MS]
To do this Adam needs to generate a JSON API dump

Dumps available at https://storage.googleapis.com/open-targets-efo-3/19.02-efo3_association_data.json.gz and https://storage.googleapis.com/open-targets-efo-3/19.02-efo3_association_data.json.gz

Analysis of diseases gained and lost
2 diseases lost when using EFO3:
EFO_0003935 - mental health, used by ChEMBL only.
Other disease has 10,340 targets associated with it. ChEMBL evidence for TSHR is all thyroid-related.
2,179 diseases gained in EFO3: the EFO IDs can be found here.
To be continued (will add the number of associations per new disease) ...

Follow-up action for @afaulconbridge specific to ePMC:

ID new papers included in this release with new EFO3 terms - do they make sense or not?

"ID new papers included in this release with new EFO3 terms - do they make sense or not?" This is not a well defined question, and therefore is not something I can answer.
In the absence of a definitive way forward, I will produce a set of ePMC evidence strings in 19.04 that have diseases that were not in 19.02 evidence strings. Hopefully this will be useful to answer the appropriate question, whatever that may be.

File avaliable at https://storage.googleapis.com/open-targets-efo-3/epmc-2019-03-27.filtered.json.gz (~36MB)

If we can sample some of the papers - 100 or so, that are identified with new EFO terms not in EFO2 , and pull out the sentences, I can check that they make sense.
Also can we count how many of the additional papers in 19.04 vs 19.02 are to new EFO3 terms in 19.04 only, how many are to old EFO2 terms in 19.02, and how many are to old EFO2 terms but new in 19.04

Key decisions:

Revert to 19.02  input file from ePMC in particular for 19.04 release and follow-up with ePMC team in advance of 19.06 release.

To ensure that existing Open Targets features are of course retained with the move to EFO 3, whilst assuring that data is updated as committed every two months.

To achieve this:
(i) we will revert to using EFO 2 for this release to allow sufficient technical time to address QC items thoroughly: #183 #578 #579
(ii) this will result in a revised release date of w/e 03May2019 - anticipated short delay of 2-3 days from our initial schedule (end April)

Since we've decided to use EFO2 for 19.04 I'm moving this to 19.06

Per meeting 17May2019:

need to analyse and ID associations which have only indirect evidence and assess what specific terms/nodes drive these volumes for EFO 3 and discuss internally w/Ian to confirm final "SLIM" list for curation by SPOT.

Discuss curation process with SPOT team in parallel.

Define steps and estimate work involved to implement SLIM EFO3.


[<IE11] Location that includes unencoded single quotation character crashes on server

Thanks for filing a bug! To save time, if you're having trouble using the library, please check off the items you have tried. If you are just asking a question, skip right to the bottom.
Please verify these steps before filing an issue, and check them off as you go

[ ✔] The relevant native JavascriptEngineSwitcher library packages are installed (such as JavaScriptEngineSwitcher.V8.Native.win-x64)
[✔ ] The VC++ 2017 runtime is installed
[ ✔] The value of SetUseReact and SetUseBabel is correct in ReactConfig.cs or Startup.cs
[ ✔] I've looked at the sample projects in this repo to verify that my app is configured correctly

I'm using these library versions:

ReactJS.NET: 5.1.2
JavaScriptEngineSwitcher: ChakraCore 3.3.0
react and react-dom: (N/A if using bundled react, or version number): ^16.9.0
webpack: (N/A if using bundled react) ^4.39.1
node: (N/A if using bundled react): 12.14.1

Runtime environment:

OS: (Mac, Windows, Linux flavor. Include 32-bit/64-bit and version)
Windows
.NET Framework or .NET Core Version:
.NET Core 3.1

Steps to reproduce

On older versions of internet explorer (IE11 below), it seems as though some characters in the url don't get encoded before sending a request to the server, specifically in this case the single quotation character ('). On other browsers (Chrome), the quotation character get's encoded before the server receives the request. Note this only happens on first initial load and/or if Javascript is disabled completely.
e.g
IE 11 Below
/?keyword=String Wit'h Single Quote
Chrome
/?keyword=String%20Wit%27h%20Single%20Quote
When using React Router, because of how the location is getting written in ReactRouterComponent and because the single quotation does not get encoded, it breaks the returned Javascript initialiser string (see line 83) and a 500 happens on the server.
https://github.com/reactjs/React.NET/blob/master/src/React.Router/ReactRouterComponent.cs#L83
This can be recreated by following the Quick start steps, open Internet Explorer 11 below and inputting a url like:
http://localhost:9457/comments?test=ttt'ttt
Potentially this also opens up a possible vulnerability maybe? Some quicks tests on IE with JS disabled, we can change the props of /comments -> initialComments by altering this url e.g:
http://localhost:9457/comments?test=tttt'},{initialComments:[{"author":{"name":"Matt","githubUsername":"mattywong"},"text":"hello world"}]},{a:'
Or alternatively display a completely different page altogether:
http://localhost:9457/comments?test=tttt'},{location:'/react-jss'},{a:'

Not too sure what's the best way to handle this. On our end (we're using an older version of React.NET), we're modifying the query string and replacing the single quote with the encoded value
...
before:
request.Path.ToString() + request.QueryString;

after:
request.Path.ToString() + request.QueryString.ToString()?.Replace("'", "%27");
...

We also looked at perhaps using a different type of encoding:
see (http://www.secretgeek.net/uri_enconding)
Though not too sure exactly the implications of above as it looks like we might get a double encoding if the request comes from Chrome or other browsers other than IE?

Definitely a bug.. thanks for the detailed report. I’ll look as soon as I can.

A fix has been shipped in 5.2.4.


Animating Threshold colors required

Gauge does not support themes of colors and animation of colors from one to another based on the changing value of the gauge while animation.

Finally this feature has been introduced. https://github.com/ashish-chopra/angular-gauge/releases/tag/v2.1.0


Limit to tv shows?

Hey, is it possible to limit this thing to tv shows only? I just started watching a movie and it tries to decode it.

Check the config options and set the one regarding movie to false

My movie config looked like the default one:
[movie]
check_credits = False
check_credits_action =
check_credits_sec = 600
credits_delay = 0
check_intro_ffmpeg_sec = 600
process_recently_added = False
process_deleted = False
create_edl = False
edl_action_type = 3
create_chapters = False

I've disabled it by parsing the metadata xml and checking what type it is.


Wallaroo rolls back to checkpoint previously aborted at ConnectorSink

Is this a bug, feature request, or feedback?
Bug
What is the current behavior?
Checkpoint #4 has been aborted at the ConnectorSink.  Later, Wallaroo elects to roll back to checkpoint #4; the ConnectorSink panics when it receives an invalid message_id in a 2PC phase 1 message.
In the CRITICAL panic message below:

offset 50666 is the sink's message_id/offset wrt the abort of checkpoint #4
The message_id 115689 is sent by Wallaroo is associated with state where checkpoint #4 had committed.

What is the expected behavior?
Proper coordination of committed & aborted ConnectorSink transactions with Wallaroo global state.
What OS and version of Wallaroo are you using?
Ubuntu Xenial/16.04 LTS + aloc-tee-source+scripts+autoscale-checkpoint+tcp-source-sink branch @ commit abb7b23f9
Steps to reproduce?
$ cd /top/of/wallaroo/repo
$ git checkout -f 7ab14750cd ; echo We need the passthrough app from another branch
$ mv examples/pony/passthrough
$ git checkout -f COMMIT-ABOVE
$ mv passthrough examples/pony
$ dd if=$HOME/wallaroo/testing/data/market_spread/nbbo/r3k-symbols_nbbo-fixish.msg bs=1000000 count=1 | od -x | sed 's/^/T/' | sed -n '1,/T3641060/p' | perl -ne 'print "\0\0\0"; print "1"; print' > /tmp/input-file.txt
$ cd testing/correctness/scripts/effectively-once
$ make -C ../../../..     PONYCFLAGS="--verbose=1 -d -Dresilience -Dtrace -Dcheckpoint_trace -Didentify_routing_ids"     build-examples-pony-passthrough build-testing-tools-external_sender     build-utils-cluster_shrinker build-utils-data_receiver     build-testing-tools-fixed_length_message_blaster

$ ./master-crasher.sh 4 crash2 crash0
...
,,c0,r0,c0,r0c2,,r2,,c2r2,c0r0,c2,r2c0,r0,,,c2c0,,r2r0,,c0,r0,c2,r2,c0,c2r2r0,,
2019-10-10 16:34:34,147 CRITICAL [aloc_sink:434] 2PC: Phase 1 invalid end_por 115689 self._out_offset 50666 file size 50666 msg2 TwoPCPhase1(txn_id='rollback--Passthrough-w-worker3-id-247008000284870596570318168641095230061:c_id=6',where_list=[(1, 105693, 115689)])
SANITY
SANITY LOOP FAILURE: pause the world

See full logs at http://wallaroolabs-dev.s3.amazonaws.com/logs/logs.20191009.1570725284.tar.gz

Note: there are two checkpoint #4s in this history, fun.
$ zegrep 'CheckpointInitiator: Checkpoint [0-9]* is complete|Injecting' tmp/wallaroo.0* | sort -t : -k 2 -n
tmp/wallaroo.0.1570725249.gz:1570725246.226691,Injecting barrier CheckpointBarrierToken(1)
tmp/wallaroo.0.1570725249.gz:1570725246.878493,CheckpointInitiator: Checkpoint 1 is complete!
tmp/wallaroo.0.1570725249.gz:1570725247.880961,Injecting barrier CheckpointBarrierToken(2)
tmp/wallaroo.0.1570725249.gz:1570725248.555408,CheckpointInitiator: Checkpoint 2 is complete!
tmp/wallaroo.0.1570725251.gz:1570725249.890674,Injecting blocking barrier CheckpointRollbackBarrierToken(Rollback 1, Checkpoint 2)
tmp/wallaroo.0.1570725251.gz:1570725250.627333,Injecting barrier CheckpointRollbackResumeBarrierToken(Rollback 1, Checkpoint 2)
tmp/wallaroo.0.1570725251.gz:1570725250.663256,Injecting barrier CheckpointBarrierToken(3)
tmp/wallaroo.0.1570725257.gz:1570725252.307384,Injecting blocking barrier CheckpointRollbackBarrierToken(Rollback 2, Checkpoint 2)
tmp/wallaroo.0.1570725257.gz:1570725254.862264,Injecting barrier CheckpointRollbackResumeBarrierToken(Rollback 2, Checkpoint 2)
tmp/wallaroo.0.1570725257.gz:1570725254.898435,Injecting blocking barrier CheckpointRollbackBarrierToken(Rollback 3, Checkpoint 2)
tmp/wallaroo.0.1570725257.gz:1570725255.641009,Injecting barrier CheckpointRollbackResumeBarrierToken(Rollback 3, Checkpoint 2)
tmp/wallaroo.0.1570725257.gz:1570725255.660805,Injecting barrier CheckpointBarrierToken(3)
tmp/wallaroo.0.1570725257.gz:1570725256.390821,Injecting blocking barrier CheckpointRollbackBarrierToken(Rollback 4, Checkpoint 2)
tmp/wallaroo.0.1570725257.gz:1570725257.130985,Injecting barrier CheckpointRollbackResumeBarrierToken(Rollback 4, Checkpoint 2)
tmp/wallaroo.0.1570725257.gz:1570725257.145936,Injecting barrier CheckpointBarrierToken(4)
tmp/wallaroo.0.1570725260.gz:1570725257.678721,Injecting blocking barrier CheckpointRollbackBarrierToken(Rollback 5, Checkpoint 2)
tmp/wallaroo.0.1570725260.gz:1570725258.404585,Injecting barrier CheckpointRollbackResumeBarrierToken(Rollback 5, Checkpoint 2)
tmp/wallaroo.0.1570725260.gz:1570725258.438930,Injecting barrier CheckpointBarrierToken(3)
tmp/wallaroo.0.1570725260.gz:1570725260.118152,Injecting blocking barrier CheckpointRollbackBarrierToken(Rollback 6, Checkpoint 2)
tmp/wallaroo.0.1570725264.gz:1570725261.287522,Injecting blocking barrier CheckpointRollbackBarrierToken(Rollback 7, Checkpoint 2)
tmp/wallaroo.0.1570725264.gz:1570725262.019931,Injecting barrier CheckpointRollbackResumeBarrierToken(Rollback 7, Checkpoint 2)
tmp/wallaroo.0.1570725264.gz:1570725262.049630,Injecting barrier CheckpointBarrierToken(3)
tmp/wallaroo.0.1570725264.gz:1570725262.531042,CheckpointInitiator: Checkpoint 3 is complete!
tmp/wallaroo.0.1570725264.gz:1570725263.535424,Injecting barrier CheckpointBarrierToken(4)
tmp/wallaroo.0.1570725268.gz:1570725266.692689,Injecting blocking barrier CheckpointRollbackBarrierToken(Rollback 8, Checkpoint 3)
tmp/wallaroo.0.1570725268.gz:1570725267.420322,Injecting barrier CheckpointRollbackResumeBarrierToken(Rollback 8, Checkpoint 3)
tmp/wallaroo.0.1570725268.gz:1570725267.458989,Injecting barrier CheckpointBarrierToken(4)
tmp/wallaroo.0.1570725268.gz:1570725267.940966,CheckpointInitiator: Checkpoint 4 is complete!
tmp/wallaroo.0.1570725272.gz:1570725269.570508,Injecting blocking barrier CheckpointRollbackBarrierToken(Rollback 9, Checkpoint 4)
tmp/wallaroo.0.1570725272.gz:1570725270.302766,Injecting barrier CheckpointRollbackResumeBarrierToken(Rollback 9, Checkpoint 4)
tmp/wallaroo.0.1570725272.gz:1570725270.330978,Injecting barrier CheckpointBarrierToken(5)
tmp/wallaroo.0.1570725272.gz:1570725271.146685,Injecting blocking barrier CheckpointRollbackBarrierToken(Rollback 10, Checkpoint 4)
tmp/wallaroo.0.1570725272.gz:1570725271.884522,Injecting barrier CheckpointRollbackResumeBarrierToken(Rollback 10, Checkpoint 4)
tmp/wallaroo.0.1570725272.gz:1570725271.922086,Injecting barrier CheckpointBarrierToken(6)
tmp/wallaroo.0.1570725272.gz:1570725272.409086,CheckpointInitiator: Checkpoint 6 is complete!
tmp/wallaroo.0:1570725273.421824,Injecting blocking barrier CheckpointRollbackBarrierToken(Rollback 11, Checkpoint 6)
tmp/wallaroo.0:1570725274.149319,Injecting barrier CheckpointRollbackResumeBarrierToken(Rollback 11, Checkpoint 6)
tmp/wallaroo.0:1570725274.185290,Injecting barrier CheckpointBarrierToken(7)

I think this is looking very dodgy.  In the wallaroo.0.1570725264.gz log file:
1570725263.535424,Injecting barrier CheckpointBarrierToken(4)
1570725266.692689,Injecting blocking barrier CheckpointRollbackBarrierToken(Rollback 8, Checkpoint 3)
1570725267.420322,Injecting barrier CheckpointRollbackResumeBarrierToken(Rollback 8, Checkpoint 3)
1570725267.458989,Injecting barrier CheckpointBarrierToken(4)
1570725267.940966,CheckpointInitiator: Checkpoint 4 is complete!

The (Rollback 8, Checkpoint 3) is in the middle of (this incarnation) checkpoint #4.  And yet after the rollback to 3 is complete, there is no new Inject message for (another) checkpoint #4.  It really looks like checkpoint #4's processing is terrible, somehow.

Note also that checkpoint #3 appears 4 times in the event log.  In this ticket's logs, we can see that we eventually do roll back to a checkpoint #3000
1570725250.685479,DEBUG,2PC,2PC: Checkpoint state 3 at ConnectorSink<PHONE_NUMBER>84870596570318168641095230061, txn-id  current_offset 50666 acked_point_of_ref 0 _pending_writev_total 0
1570725255.672551,DEBUG,2PC,2PC: Checkpoint state 3 at ConnectorSink<PHONE_NUMBER>84870596570318168641095230061, txn-id  current_offset 50666 acked_point_of_ref 0 _pending_writev_total 0
1570725257.159181,DEBUG,2PC,2PC: Checkpoint state 4 at ConnectorSink<PHONE_NUMBER>84870596570318168641095230061, txn-id  current_offset 50666 acked_point_of_ref 0 _pending_writev_total 0
1570725258.455899,DEBUG,2PC,2PC: Checkpoint state 3 at ConnectorSink<PHONE_NUMBER>84870596570318168641095230061, txn-id  current_offset 50666 acked_point_of_ref 0 _pending_writev_total 0
1570725262.069520,DEBUG,2PC,2PC: Checkpoint state 3 at ConnectorSink<PHONE_NUMBER>84870596570318168641095230061, txn-id  current_offset 50666 acked_point_of_ref 0 _pending_writev_total 0

Here's the rollback entry for CP 3, as logged by initializer.  Note the timestamp: it is 4.63 seconds after the 4th checkpoint entry was written.
1570725266.692689,Injecting blocking barrier CheckpointRollbackBarrierToken(Rollback 8, Checkpoint 3)

@jtfmumm: Possible fix @ commit 2739471f3, thoughts?

Note to self: this appears to be a very common crash, as run with the suggested ./master-crasher.sh 4 crash2 crash0.

@slfritchie I think the problem could be that we never rethought checkpoint id assignment in light of the addition of checkpoint abort support.  It seems to me that the checkpoint id should be incremented both in the success and failure case, but we are only doing it in the success case. I think we should look at both the checkpoint ids as signifying rounds (as we do with the rollback ids).  So what we are calling _last_complete_checkpoint_id in CheckpointInitiator should mean the last completed round, not necessarily the last successful checkpoint.  An abort completes a round, even if it doesn’t involve committing a checkpoint.

This of course means that we need to represent two separate concepts: (1) last successful checkpoint that we can roll back to and (2) current round id. So we'll need to have discussions around this before implementing a fix.

Seen again today on today's master branch at commit 4323b71 + connector-sink-fixes branch/PR #3042 using ./master-crasher.sh 2 crash0 crash1 crash-sink, see logs at http://wallaroolabs-dev.s3.amazonaws.com/logs/logs.1573071315.tar.gz.
Noteworthy:
$ zgrep 'Checkpoint Barrier CheckpointBarrierToken.*Complete' /tmp/wallaroo.0.* /tmp/wallaroo.0
/tmp/wallaroo.0.1573071008.gz:1573071005.194525,Checkpoint_Initiator: Checkpoint Barrier CheckpointBarrierToken(63) Complete
/tmp/wallaroo.0.1573071008.gz:1573071006.715433,Checkpoint_Initiator: Checkpoint Barrier CheckpointBarrierToken(64) Complete
/tmp/wallaroo.0.1573071008.gz:1573071008.095921,Checkpoint_Initiator: Checkpoint Barrier CheckpointBarrierToken(66) Complete
/tmp/wallaroo.0.1573071013.gz:1573071011.160774,Checkpoint_Initiator: Checkpoint Barrier CheckpointBarrierToken(65) Complete
/tmp/wallaroo.0.1573071013.gz:1573071012.671681,Checkpoint_Initiator: Checkpoint Barrier CheckpointBarrierToken(66) Complete
/tmp/wallaroo.0              :1573071016.352944,Checkpoint_Initiator: Checkpoint Barrier CheckpointBarrierToken(67) Complete
/tmp/wallaroo.0              :1573071017.879442,Checkpoint_Initiator: Checkpoint Barrier CheckpointBarrierToken(68) Complete


ValueId as String

Should this be merged alreday?

I don't think so, marked as do not merge for now


JS assets preload

Implementation of modern features "assets preload" from w3 draft https://www.w3.org/TR/preload/
It works only on latest Chrome browser

waiting for community review

I close ... after many tests it is difficult to release a stable version. Better to wait to have a better idea.


build landing page to route users between eng and span surveys

On which service? How do users get to the landing page?

this is for the web surveys, currently we're using cfa's paid version of
typeform that allows us to put logic into the survey and route the user to
english or spanish, but the free version doesn't have logic, an intro page,
or allow you to send users somewhere (the dashboard) once the survey has
been submitted.
so for the county to use the free version, we need to create an english
survey and a spanish survey, and have each linked to a button on this page,
who's url will be c4a.me/picsurvey
On Wed, Sep 9, 2015 at 10:27 AM, Sophia Dengo<EMAIL_ADDRESS>wrote:

On which service? How do users get to the landing page?
—
Reply to this email directly or view it on GitHub
https://github.com/codeforamerica/mdc-feedback/issues/272#issuecomment-138983730
.

Ok. Makes sense. However, I’m pushing it to the next sprint — this is way in the future still.
On Wed, Sep 9, 2015 at 12:07 PM, Mathias Gibson<EMAIL_ADDRESS>wrote:

this is for the web surveys, currently we're using cfa's paid version of
typeform that allows us to put logic into the survey and route the user to
english or spanish, but the free version doesn't have logic, an intro page,
or allow you to send users somewhere (the dashboard) once the survey has
been submitted.
so for the county to use the free version, we need to create an english
survey and a spanish survey, and have each linked to a button on this page,
who's url will be c4a.me/picsurvey
On Wed, Sep 9, 2015 at 10:27 AM, Sophia Dengo<EMAIL_ADDRESS>wrote:

On which service? How do users get to the landing page?
—
Reply to this email directly or view it on GitHub
https://github.com/codeforamerica/mdc-feedback/issues/272#issuecomment-138983730
.

Reply to this email directly or view it on GitHub:
https://github.com/codeforamerica/mdc-feedback/issues/272#issuecomment-139013566

v3 draft in spanish - https://miamidadecounty.typeform.com/to/eEHiJU
v3 draft in english - https://miamidadecounty.typeform.com/to/F7hSOx

it would also be nice to have a finishing page if the county isnt aren't paying for typeform, bc currently hitting submit sends them to the below. That seems much harder, if possible. @ErnieAtLYD have you heard anything from Mike on that?

http://<IP_ADDRESS>:9000/choose-survey/


resource_auditor: add audit for HEAD default branch

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.

[x] Have you successfully run brew typecheck with your changes locally?
[x] Have you successfully run brew tests with your changes locally?

This PR adds an audit to ensure that the correct default branch for a Git repository is specified for head when using --strict --online. For main and other alternatives, we need to explicitly specify the default branch using branch:, as the default is master.
Adding an audit will ensure this is checked in PRs to add new formulae.
I haven't written a test for this audit as it would require network access (and use a real repository URL), but I'd be happy to add one if that's preferable.
(CC @vladimyr)

Review period will end on 2021-07-15 at 18:00:14 UTC.

I realised that for some formulae we want the specified branch to not be the default branch, for example, for versioned formulae where different versions could have different development branches. I don't think this would be the case for too many formulae, so I think I'll add an allowlist for the audit, along with the suggested changes.

Review period ended.

I realised that for some formulae we want the specified branch to not be the default branch – like versioned formulae where different versions have different development branches. I don't think this would be the case for too many formulae, so I think I'll add an allowlist for the audit, along with the suggested changes.

Yeah, this is definitely a good idea to have. I'd say go ahead and continue with this route and if it turns out there are a ton of formulae on the list, we can change the audit to just be "you need to specify a branch name"

Could this be done for stable, too?
Also, I'm tempted to not make this a normal audit over --strict. Would that be unreasonable because of the number of formulae that would need to be changed? What about making the audit is --strict if the default branch is master and not --strict otherwise (and then we can plan on removing the --strict in the future once more formulae are updated.

Could this be done for stable, too?

Stable URLs always specify a revision and mostly have a tag, so I don't know if this would be necessary. Hope there isn't some case I'm forgetting about here.

Would that be unreasonable because of the number of formulae that would need to be changed?

I haven't yet checked the number of formulae that need to be updated, but I thought keeping this as a strict audit would be better because:

Failing CI for an effectively syntax-only change seems undesirable. Those who use HEAD-installs and other interested contributors could volunteer to make the change in a separate syntax-only PR anyway.
It will still be checked for brew audit --new, so new formulae have the correct default branch specified.

Seems like specifying the branch anyway (whether it's master or not) is better in the long run. I'll tweak this so that a branch must be specified, and add an allowlist which we would use when the default branch is intentionally not used for the formula (versioned formula with head and other cases).

Apologies for the late update, the latest commit makes specifying a branch mandatory for head and also adds an allowlist for specifying formulae where we intentionally use non-default branches. I tested this with some formulae:

sdl_mixer for an intentional non-default branch.
zsh-vi-mode to ensure that specifying a branch is mandatory.
Modifying the head branch to a formula and ensuring the correct default branch is reported by the audit.

The allowlist, located at audit_exceptions/head_non_default_branch_allowlist.json in a Tap would look like this:
{
    "sdl_mixer": "SDL-1.2",
    ...
}

Only one issue remains unsolved – head resources that specify non-default branches intentionally. I'll need to look for examples but here's a hypothetical one:
head do
  ...

  resource "example-resource" do
    url "https://github.com/example-resource/example-resource.git", branch: "example"
  end
end

At the moment, the audit would raise an error if example isn't the default branch. We'll have to extend the allowlist to account for such cases as well, but I'm not sure what a good format/schema would be, looking forward to your suggestions. Here's one potential format which looks complicated, but such formulae should be very rare:
{
    "sdl_mixer": "SDL-1.2",
    "example-formula": {
        "branch": "dev",
        "resources": {
            "example-resource": "example"
        }
    }
}

@nandahkrishna Just for the record I still feel bad sending you down the rabbit hole. I really didn't mean to and I honestly wish I was able to help you.
Here goes a big thank you to @nandahkrishna and the rest of the team! Great job, as usual! ❤

No problem at all @vladimyr, happy to have worked on this 😄.


OS Support

What OSes does this software support currently? Some details about this in the README would be appreciated.

It's based on Electron, so I expect it to work everywhere. But I tested it only on MacOS, so I prebuild dmg files only. And yes, I think I'll add this information to the README.

On my WSL 2 in Windows 10, I get the error below when I try the build instructions:

I have never used yarn before, so I could be making a very obvious mistake.

@chintanp Please try to use this instruction

@apsavin thanks for the link. Should have googled before posting. I tried the above and got yarn installed and working (i.e. it is doing its nodejs things). However, then stuck on the error below:
yarn && yarn package
yarn install v1.22.5
[1/4] Resolving packages...
success Already up-to-date.
$ electron-builder install-app-deps
  • electron-builder  version=22.9.1
  • loaded configuration  file=package.json ("build" field)
  • rebuilding native dependencies<EMAIL_ADDRESS>platform=linux arch=x64
  • install prebuilt binary  name=keytar version=4.13.0 platform=linux arch=x64
  • build native dependency from sources  name=keytar
                                          version=4.13.0
                                          platform=linux
                                          arch=x64
                                          reason=prebuild-install failed with error (run with env DEBUG=electron-builder to get more information)
                                          error=prebuild-install info begin Prebuild-install version 5.3.0
    prebuild-install info looking for cached prebuild @ /root/.npm/_prebuilds/1d9a41-keytar-v4.13.0-electron-v75-linux-x64.tar.gz
    prebuild-install http request GET https://github.com/atom/node-keytar/releases/download/v4.13.0/keytar-v4.13.0-electron-v75-linux-x64.tar.gz
    prebuild-install http 404 https://github.com/atom/node-keytar/releases/download/v4.13.0/keytar-v4.13.0-electron-v75-linux-x64.tar.gz
    prebuild-install WARN install No prebuilt binaries found (target=7.3.3 runtime=electron arch=x64 libc= platform=linux)

  • rebuilding native dependency  name=keytar version=4.13.0
Done in 8.35s.
yarn run v1.22.5
$ yarn build && electron-builder build --publish never
$ concurrently "yarn build-main" "yarn build-renderer"
$ cross-env NODE_ENV=production webpack --config ./configs/webpack.config.renderer.prod.babel.js --colors
$ cross-env NODE_ENV=production webpack --config ./configs/webpack.config.main.prod.babel.js --colors
[0] /mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/cli.js:281
[0]                             throw err;
[0]                             ^
[0]
[0] Error: Cannot find module 'vs/editor/contrib/gotoSymbol/goToCommands'
[0] Require stack:
[0] - /mnt/d/temp/electronapps/pgrights/node_modules/monaco-editor-webpack-plugin/out/index.js
[0] - /mnt/d/temp/electronapps/pgrights/configs/webpack.config.base.js
[0] - /mnt/d/temp/electronapps/pgrights/configs/webpack.config.main.prod.babel.js
[0] - /mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/utils/convert-argv.js
[0] - /mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/cli.js
[0] - /mnt/d/temp/electronapps/pgrights/node_modules/webpack/bin/webpack.js
[0]     at Function.Module._resolveFilename (internal/modules/cjs/loader.js:880:15)
[0]     at Function.resolve (/mnt/d/temp/electronapps/pgrights/node_modules/v8-compile-cache/v8-compile-cache.js:164:23)
[0]     at resolveMonacoPath (/mnt/d/temp/electronapps/pgrights/node_modules/monaco-editor-webpack-plugin/out/index.js:34:28)
[0]     at /mnt/d/temp/electronapps/pgrights/node_modules/monaco-editor-webpack-plugin/out/index.js:173:63
[0]     at Array.map (<anonymous>)
[0]     at createLoaderRules (/mnt/d/temp/electronapps/pgrights/node_modules/monaco-editor-webpack-plugin/out/index.js:173:43)
[0]     at MonacoEditorWebpackPlugin.apply (/mnt/d/temp/electronapps/pgrights/node_modules/monaco-editor-webpack-plugin/out/index.js:91:23)
[0]     at webpack (/mnt/d/temp/electronapps/pgrights/node_modules/webpack/lib/webpack.js:51:13)
[0]     at processOptions (/mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/cli.js:272:16)
[0]     at /mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/cli.js:364:3
[0]     at Object.parse (/mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/node_modules/yargs/yargs.js:576:18)
[0]     at /mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/cli.js:49:8
[0]     at Object.<anonymous> (/mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/cli.js:366:3)
[0]     at Module._compile (internal/modules/cjs/loader.js:1063:30)
[0]     at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
[0]     at Module.load (internal/modules/cjs/loader.js:928:32)
[0]     at Function.Module._load (internal/modules/cjs/loader.js:769:14)
[0]     at Module.require (internal/modules/cjs/loader.js:952:19)
[0]     at require (internal/modules/cjs/helpers.js:88:18)
[0]     at Object.<anonymous> (/mnt/d/temp/electronapps/pgrights/node_modules/webpack/bin/webpack.js:156:2)
[0]     at Module._compile (internal/modules/cjs/loader.js:1063:30)
[0]     at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
[0]     at Module.load (internal/modules/cjs/loader.js:928:32)
[0]     at Function.Module._load (internal/modules/cjs/loader.js:769:14)
[0]     at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
[0]     at internal/main/run_main_module.js:17:47 {
[0]   code: 'MODULE_NOT_FOUND',
[0]   requireStack: [
[0]     '/mnt/d/temp/electronapps/pgrights/node_modules/monaco-editor-webpack-plugin/out/index.js',
[0]     '/mnt/d/temp/electronapps/pgrights/configs/webpack.config.base.js',
[0]     '/mnt/d/temp/electronapps/pgrights/configs/webpack.config.main.prod.babel.js',
[0]     '/mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/utils/convert-argv.js',
[0]     '/mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/cli.js',
[0]     '/mnt/d/temp/electronapps/pgrights/node_modules/webpack/bin/webpack.js'
[0]   ]
[0] }
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
[0] yarn build-main exited with code 1
[1] /mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/cli.js:281
[1]                             throw err;
[1]                             ^
[1]
[1] Error: Cannot find module 'vs/editor/contrib/gotoSymbol/goToCommands'
[1] Require stack:
[1] - /mnt/d/temp/electronapps/pgrights/node_modules/monaco-editor-webpack-plugin/out/index.js
[1] - /mnt/d/temp/electronapps/pgrights/configs/webpack.config.base.js
[1] - /mnt/d/temp/electronapps/pgrights/configs/webpack.config.renderer.prod.babel.js
[1] - /mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/utils/convert-argv.js
[1] - /mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/cli.js
[1] - /mnt/d/temp/electronapps/pgrights/node_modules/webpack/bin/webpack.js
[1]     at Function.Module._resolveFilename (internal/modules/cjs/loader.js:880:15)
[1]     at Function.resolve (/mnt/d/temp/electronapps/pgrights/node_modules/v8-compile-cache/v8-compile-cache.js:164:23)
[1]     at resolveMonacoPath (/mnt/d/temp/electronapps/pgrights/node_modules/monaco-editor-webpack-plugin/out/index.js:34:28)
[1]     at /mnt/d/temp/electronapps/pgrights/node_modules/monaco-editor-webpack-plugin/out/index.js:173:63
[1]     at Array.map (<anonymous>)
[1]     at createLoaderRules (/mnt/d/temp/electronapps/pgrights/node_modules/monaco-editor-webpack-plugin/out/index.js:173:43)
[1]     at MonacoEditorWebpackPlugin.apply (/mnt/d/temp/electronapps/pgrights/node_modules/monaco-editor-webpack-plugin/out/index.js:91:23)
[1]     at webpack (/mnt/d/temp/electronapps/pgrights/node_modules/webpack/lib/webpack.js:51:13)
[1]     at processOptions (/mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/cli.js:272:16)
[1]     at /mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/cli.js:364:3
[1]     at Object.parse (/mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/node_modules/yargs/yargs.js:576:18)
[1]     at /mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/cli.js:49:8
[1]     at Object.<anonymous> (/mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/cli.js:366:3)
[1]     at Module._compile (internal/modules/cjs/loader.js:1063:30)
[1]     at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
[1]     at Module.load (internal/modules/cjs/loader.js:928:32)
[1]     at Function.Module._load (internal/modules/cjs/loader.js:769:14)
[1]     at Module.require (internal/modules/cjs/loader.js:952:19)
[1]     at require (internal/modules/cjs/helpers.js:88:18)
[1]     at Object.<anonymous> (/mnt/d/temp/electronapps/pgrights/node_modules/webpack/bin/webpack.js:156:2)
[1]     at Module._compile (internal/modules/cjs/loader.js:1063:30)
[1]     at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
[1]     at Module.load (internal/modules/cjs/loader.js:928:32)
[1]     at Function.Module._load (internal/modules/cjs/loader.js:769:14)
[1]     at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
[1]     at internal/main/run_main_module.js:17:47 {
[1]   code: 'MODULE_NOT_FOUND',
[1]   requireStack: [
[1]     '/mnt/d/temp/electronapps/pgrights/node_modules/monaco-editor-webpack-plugin/out/index.js',
[1]     '/mnt/d/temp/electronapps/pgrights/configs/webpack.config.base.js',
[1]     '/mnt/d/temp/electronapps/pgrights/configs/webpack.config.renderer.prod.babel.js',
[1]     '/mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/utils/convert-argv.js',
[1]     '/mnt/d/temp/electronapps/pgrights/node_modules/webpack-cli/bin/cli.js',
[1]     '/mnt/d/temp/electronapps/pgrights/node_modules/webpack/bin/webpack.js'
[1]   ]
[1] }
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
[1] yarn build-renderer exited with code 1
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Initial research on Google did not help much with this.


Fixing eslint rule errors

ref #25
As a developer, I want to fix the errors from eslint in order of priority, so that we can get the health of the codebase to a manageable level.
DoD:

eslint errors fixed in order of priority
a list of errors that are difficult/impossible to fix and the reason why

The three big ones are:
"no-undef": "off"
"no-fallthrough": "off"
"no-duplicate-case": "off"


Revert node ical update

With my last dependency update node-ical was updated from 0.13.0 to 0.14.1.
Starting mm with the calendar module results now in module luxon not found when using npm install --no-optional.
So reverting this, must be solved on the node-ical side.

Codecov Report

Merging #2716 (aed0056) into develop (559970c) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop    #2716   +/-   ##
========================================
  Coverage    69.46%   69.46%
========================================
  Files            8        8
  Lines          262      262
========================================
  Hits           182      182
  Misses          80       80

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 559970c...aed0056. Read the comment docs.

changelog check fails but this time I didn't forget it ...


[Info] Issues Organization

Issues Organization
I am not a big fan of formalisation. However, we are a community with the same target to provide smart and high-level API for spreadsheet files, a community built from a random person who works together to create awesome toolset. I write this issue to explain how I organise issues here - please do not track this as strict rules_ :)
Flow:

The [Proposal] is an issue created for giving some proposition or share with some idea. I often create this because of two reasons: do not forgive about it and get information for you, that I am want to do things like that somewhere. All proposals are tracks as described in #144.

The [Draft] - this is an issue, where we will attempt to precise what should happen, what should be done (sometimes I propose how). I am counting for the discussion and validation of ideas on this step.

The accepted (label) - ready to do, these issues haven't any [prefix]

The important (label) -  issues in my opinion really worth to do first (I'm trying to write why in an issue -if I forget, ask why)

The not ready yet (label) - usually drafts but not always (not a draft when part of the issue is ready to do or done with other issues). I'm actively working about precise these issues.

The good first issue (label) - really well-described issues with step by step instruction.

The '[Info]' - issue created just for giving an information

"Ok, fine, but what profit it gives?"

I intend to enable to smart filtering issues by labels:

Easy to do: good first issue

Ready to do: accepted Issues

Important (setting course) Important

Nearly: not ready yet

Provides guidelines

Self-descriptive issue titles

I've added project xlsx-import: https://github.com/Siemienik/XToolset/projects/1

Current project board: https://github.com/users/Siemienik/projects/2/views/1


java.lang.IncompatibleClassChangeError

Click to expand!
Issue Type
Build/Install
Source
binary
Tensorflow Version
tf 2.8.0
Custom Code
No
OS Platform and Distribution
Android
Mobile device
Android 12
Python version
No response
Bazel version
No response
GCC/Compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current Behaviour?
An Android App is crashing.

I have Android App installed on a device that uses third-party SDK which is internally using tfLite. When I update the tfLite version from 2.5.0 to 2.8.0 in my app and try to install it, I am getting a crash.

Caused by: java.lang.IncompatibleClassChangeError: Found interface org.tensorflow.lite.Tensor, but class was expected (declaration of 'org.tensorflow.lite.Tensor' appears in /data/data/com.m.f.dev/code_cache/.overlay/base.apk/classes3.dex)
at com.spl.sdk.ondevicefull.k.a(SourceFile:2031)
        at com.spl.sdk.ondevicefull.k.b(SourceFile:43)
        at com.spl.sdk.ondevicefull.k.a(SourceFile:24)
        at com.spl.sdk.ondevicefull.g.c(SourceFile:2284)
        at com.spl.sdk.ondevicefull.g.d(SourceFile:382)
        at com.spl.sdk.ondevicefull.g$a.a(SourceFile:404)
        at com.spl.sdk.events.d$4.run(SourceFile:502)
        at ....

I was just wondering if you could provide me with some pointers. If there is any change happening in the source code that might be producing this crash.

Standalone code to reproduce the issue
I don't know how I can provide a steps to reproduce this crash, since I don't have access to the third party library code.

Relevant log output
Caused by: java.lang.IncompatibleClassChangeError: Found interface org.tensorflow.lite.Tensor, but class was expected (declaration of 'org.tensorflow.lite.Tensor' appears in /data/data/com.m.f.dev/code_cache/.overlay/base.apk/classes3.dex)
at com.spl.sdk.ondevicefull.k.a(SourceFile:2031)
        at com.spl.sdk.ondevicefull.k.b(SourceFile:43)
        at com.spl.sdk.ondevicefull.k.a(SourceFile:24)
        at com.spl.sdk.ondevicefull.g.c(SourceFile:2284)
        at com.spl.sdk.ondevicefull.g.d(SourceFile:382)
        at com.spl.sdk.ondevicefull.g$a.a(SourceFile:404)
        at com.spl.sdk.events.d$4.run(SourceFile:502)
        at ....

Hi @MobileLearner003 ! Above Error points towards incompatibility of third party SDK with 2.8 Lite library? Could you share the steps you followed . Thank you!

Thanks @MobileLearner003 for the update. @sachinprasadhs! Could you please look at this issue. Thank you!

Hi @DharitParmar
Could you please check in the latest version and see if the issue still persists?
Providing standalone code or a toy tflite model will help us better understand and debug the issue.
Thanks.


Mint $nice tokens when someone create eligible donations on some specific projects

Now givback-calculation get donations data, and @divine-comedian fetchs data from that and distribute/mint $nice tokens for donors, but we would move it to impact-graph
If a donation to the giveth project is made in certain stablecoins - once it is confirmed we mint automatically the tokens
I need to know how to call smart contract function for this purpose (@divine-comedian  would setup an address that we can give the minter permission to) then pass the private key to the impact-graph and allow it to call mint for the $nice token

This is where on the UI we can give permission to a specified ETH address to have minting privileges for $nice
https://xdai.aragon.blossom.software/#/nrgiv/permissions/app/0x80446f514aaec7cfb7eb791830a672c5ef047943
I will ask amin the process for securely generating a new account we can use for minting from the impact-graph

Generally, we have two ways
Hold the key in our application and do the transactions! less research and hard to maintenance!!
Relying on a third-party service which I am not sure will work for us, but it's worth giving it a shot
These third-party services may help
https://www.openzeppelin.com/defender
https://docs.gelato.network/introduction/what-is-gelato

Defender Relay service seems to be right up to our alley https://docs.openzeppelin.com/defender/relay

Defender Relay service seems to be right up to our alley https://docs.openzeppelin.com/defender/relay

Also supports all networks we use

Openzeppelin has a webhook trigger that enables us to send transaction and monitor it. It has versatile features like replacing/rejecting blocked transactions and tracking the state of a sent transaction.
Gelato hasn't implemented webhook feature yet

Coming back around to this feature. I tested out OP Defender and managed to setup a relayer instance in the dapp and make a local call to the relayer which then minted a token to a specified address using an aragon DAO token-manager.
You can check it out here:
https://github.com/divine-comedian/defender-autotask-examples/blob/master/typescript/src/index.ts#L11
I think we can implement this without too much headache.
@jainkrati maybe this is a good issue for training some devs who are interested in BE work.
For context:
Currently in the dapp when a user donates to the giveth project on giveth in stablecoins (USDC, DAI, USDT etc..) we give them $nice tokens - these $nice tokens are used to buy swag from our shop and are an extra thank you for donating stable tokens.
However this is done manually every 2 weeks according to GIVbacks rounds and collecting the list of eligible receipients, making the votes to send the tokens etc.. all of it is done manually! So the idea here is that we could do it automatically using a service like OP Defender which can recevier HTTPS requests and make authenticated calls and trigger smart contract interactions from an EOA relayer address that Open Zeppelin facilitates our control over.
In order to make this happen we would need to:

setup relayer addresss in Open Zeppelin (or use the one I made)
import API keys to BE repo
redefine list of eligible stables (hardcoded or from AdminJS)
Give permissions on Aragon DAO for relayer to mint $nice tokens

then on donation of eligible stable tokens:

check stable matches one of the eligible tokens
confirm tx was successful
get data for hashing mint data (donor address, amount donated)
hash data by calling mint function with arguments (receiver, amount)
send callData to relayer (defining target as token-manager on Aragon DAO) via authenticated https request

from there the relayer forwards the callData to aragon DAO and mints the $nice tokens to the donor on gnosis chain
we would also need to keep an eye on the relayer's balance since I think we do have to top it up manually with gas

Well @RamRamez is all set with his BE skills and can take this up for progressing along the learning curve!!


Replace yiisoft/yii-view with yiisoft/yii-view-renderer in composer dependency

What steps will reproduce the problem?
As suggested here https://github.com/yiisoft/demo/issues/621, please replace yiisoft/yii-view with yiisoft/yii-view-renderer
What is the expected result?
What do you get instead?
Additional info

Q
A

Version
1.0.?

PHP version

Operating system

code fixed: https://github.com/yiisoft/yii-swagger/pull/97
dependency fixed: https://github.com/yiisoft/yii-swagger/pull/103
Release for fixes: https://github.com/yiisoft/yii-swagger/releases/tag/2.1.0, https://github.com/yiisoft/yii-swagger/releases/tag/2.1.1


Empty token sorts aren't treated as empty

module TEST
    syntax Foo [token]
    syntax Bar [token]
    syntax Foo ::= Bar [token]

    rule F => .K
endmodule

results in
$ kompile test.k --backend llvm
[Error] Compiler: Had 1 parsing errors.
[Error] Inner Parser: Parsing ambiguity.
1: syntax Foo ::= Bar [token]
    #token(Foo,"F")
2: syntax K ::= K ":K" [format(%1%2), org.kframework.kore.Sort(K)]
    #SemanticCastToK(#token("F","#KVariable"))
	Source(/home/njr/co/github.com/kframework/matching-logic-prover/prover/./test.k)
	Location(6,10,6,11)

I believe what is happening here is that F is parsing as a variable of sort Bar and then becomes a token of sort Foo due to the production with the token attribute. It's possible one could argue that the [token] attribute should not convert terms containing variables, but it's not exactly clear how one would implement that. What exactly are you trying to do? It may be simpler to work around this particular problem.

I see. Makes sense.
I had the following definition:
module TOKENS
  // Lexical
  syntax UpperName
  syntax LowerName ::= CheckSATResult
  syntax SharpName
  syntax ColonName
  syntax PipeQID
  syntax Decimal

  // Abstract
  syntax Symbol ::= UpperName | LowerName
  syntax VariableName ::= UpperName
  syntax Sort ::= UpperName | LowerName
  ....
  syntax SMTLIB2SimpleSymbol ::= UpperName | LowerName | SMTName
endmodule

that I was porting to the LLVM backend. I fixed it by moving the UpperName/LowerName to the SYNTAX module


It would be helpful to show how the feed groups for this example are set up in the dashboard.

Thank you for your feedback. We recently changed the default feed groups for new apps in our dashboard. I will make sure to update the feed groups used in this example to reflect these changes in the near future.


Adding mylar

Mylar is a comic book dowloader https://github.com/mylar3/mylar3

Thanks for the PR! Keep em coming!


Extension to hourly components?

First, prophet is super cool! I'm already playing around with some health related data. I have intraday data, but it looks like prophet doesn't really care, and only wants daily data. Has there been discussion of allowing for intraday data?

The R version is that we use the zoo Date type for the ds column, which only has day resolution. Our focus has so far been on daily data and so that was the most straightforward approach, but we are definitely interested in making this work for hour data.
Besides updating the R datatype to be a datetime (Python already is pd.datetime), there are two other parts of the code that assume daily data: holidays and seasonality.
Holidays essentially join on date to identify the rows in the history/future that should get the feature for that holiday. We have to decide how we want to think about holidays with hourly data.
Seasonality features are done by converting the dates to number of days since Unix epoch. The way this is done right now provides only differences in the (integer) number of days; for hourly data that needs to be changed to get the fractional component.

It would be really great if you  can extend it to work on lower (hourly, minutely) temporal granularity.

When is this extension available for R users?.
Thank you very much.

I took at stab at extending prophet to support daily seasonality by looking at hourly components as well: https://github.com/hellais/prophet/tree/feature/daily-seasonality.
I am not entirely sure if 3 is the correct number of components for the fourier series.

Are hourly and half-hourly predictions supported in the end?

hourly predictions is coming soon ?

@benoitkoen @ofir887 I opened PR#135 with support for hourly predictions for the python library. It's missing R support before it can be merged into master as I have limited R knowledge someone else would have to write that.

This is done in the v0.2 branch, for Python. Thanks to @hellais for kicking it off.
If you have sub-daily data, try it out! Example here:
https://github.com/facebookincubator/prophet/blob/v0.2/notebooks/non-daily_data.ipynb

@bletham  great work to handle non-daily data. Just want to make sure. How is holiday handled for hourly data? How is holiday handled for monthly data?

@mucun1988 this was something we debated a bit. With daily data, holidays provide a constant shift on the holiday date. With sub-daily data, this behavior has currently not been changed, meaning, there is a constant shift across the entire holiday date. This is clearly not going to work all the time, you can imagine that you might want a holiday effect that peaks during the day and returns to baseline at the end of the day. But I think that this will be a reasonable approximation for now.
You can imagine fitting a holiday-specific daily seasonality term, basically by replacing the current holiday regressor [0, 0, 1, 0, ...] with something that looks like the daily seasonality regressors, but is zero off-holiday. #101 will soon allow for entirely custom external regressors, which could be used to do this manually. We could build functionality in the future for constructing these automatically, but I can think of a few ways to do this (individual daily curves for each holiday? daily curve for all holidays with a multiplier specific to each holiday?) and I think it would be problem-specific which is appropriate.
Monthly data are treated like daily data (the model is continuous-time), so intra-month holidays would best be specified as being on whatever day of the month the past/future dates are labeled with.

@bletham  Monthly data are treated like daily data (the model is continuous-time), so intra-month holidays would best be specified as being on whatever day of the month the past/future dates are labeled with.
So suppose my data in monthly and I would like model thanksgiving holiday, then it does not matter I specify thanksgiving as Nov 1 or Nov 20 (as long as some date in Nov). Do I understand correctly?

You would want to specify it on whatever day in your history has the effect from Thanksgiving. So if the input data are monthly like:
ds, y
2016-10-01, 20
2016-11-01, 30
2016-12-01, 40

and the row with 2016-11-01 represents the entire month of Nov, then you'd want to put the holiday on 2016-11-01, and on 11-01 moving forward.
Then when making the dataframe of dates for future predictions, just make sure it is also the first of the month, like
ds
2017-10-01
2017-11-01
2017-12-01

Does that make sense?

@bletham  that makes sense and many thanks!

Fitting sub-daily data is now finished in R in the v0.2 branch. A big thanks to @qwng who went through the painful process of converting all of the date types to POSIXct. If you have sub-daily data try it out!
devtools::install_github('facebookincubator/prophet', subdir='R', ref='v0.2')

Awesome, thank you! Have been awaiting this feature for some time.

@bletham just in time to be included in my master thesis (printing on Saturday), good job hahaha

Hi @bletham and @seanjtaylor, Thanks for sharing this fantastic contribution. If I understand correctly, the current holiday regressor (in make_holiday_features) is returning a binary vector [0, 0, 1, 0, ...] only because scale_ratio = (self.holidays_prior_scale = 10) / (self.seasonality_prior_scale = 10)
Is it a problem if the 'expanded_holidays' would contain instead non integer negative values?
And, Would it be sensible to extend the ‘expanded_holidays’ DataFrame with other features than holidays? That is, having in seasonal_features an arbitrary number of columns to be appended relating to other variables than 'holidays'.

@CarlosVaquero this is coming in #101, within the next week.

Hi @bletham , is there any estimated date when v0.2 is going to be released?
Thanks,
Oscar

There is definitely still a need for adding hourly seasonality to python's fbprophet!

This is done in v0.2 that is now available on CRAN and pypi. Check out the documentation here:
https://facebookincubator.github.io/prophet/docs/non-daily_data.html
And if you run into any issues feel free to reopen here or make a new task.

ds
y

2017-07-28 09:30:00
5012

2017-07-28 09:30:01
3582

2017-07-28 09:30:02
3205

2017-07-28 09:30:03
2680

... ...
... ...

2017-07-28 10:30:00
4510

2017-08-28 09:30:00
6831

2017-08-28 09:30:01
4370

2017-08-28 09:30:02
3196

2017-08-28 09:30:03
2642

... ...
... ...

2017-08-28 10:30:00
1862

2017-09-28 09:30:00
4382

2017-09-28 09:30:01
5542

2017-09-28 09:30:02
2572

2017-09-28 09:30:03
2576

... ...
... ...

2017-09-28 10:30:00
2770

@bletham

Hi,
I am currently trying to use Prophet to predict flows in a network. The data like this:

ds
y

2017-07-28 09:30:00
5012

2017-07-28 09:30:01
3582

2017-07-28 09:30:02
3205

2017-07-28 09:30:03
2680

... ...
... ...

2017-07-28 10:30:00
4510

2017-08-28 09:30:00
6831

2017-08-28 09:30:01
4370

2017-08-28 09:30:02
3196

2017-08-28 09:30:03
2642

... ...
... ...

2017-08-28 10:30:00
1862

2017-09-28 09:30:00
4382

2017-09-28 09:30:01
5542

2017-09-28 09:30:02
2572

2017-09-28 09:30:03
2576

... ...
... ...

2017-09-28 10:30:00
2770

I want to predict the next period data--- '2017-10-28 9:30:00 10:30:00'. But I saw the Prophet can’t predict secondly data. I have tried to adapt 'forecaster.py' ,but it can't work.
I need your help,please.
@bletham

Hi, @kikizxd
I've also been trying to use Prophet to detect anomalies in secondly data and have not been able to.
Have you had any progress with this?
Thanks.

@azriele If you have v0.2 it should be working with secondly data, what is the particular issue you're running into?

@bletham Great!
For some reason I had v 0.1.1 and I just upgraded it to 0.2
So as far as I can understand I can use secondary data and weekly seasonality?
Thanks.

Yep!

@bletham The link to "Non Daily Data" above isn't working b/c facebookincubator.github.io change to facebook.github.io
Here's a working link - https://facebook.github.io/prophet/docs/non-daily_data.html

Thanks, that's a shame the URL forwarding doesn't include subdirectories.


WP Plugin stop working when upgrade to 5.3.1

Fatal error: Uncaught Error: Class "Puc_v4_Factory" not found in /wp-content/plugins/bs-swiper-main/main.php:21 Stack trace: #0 /wp-admin/includes/plugin.php(2318): include_once() #1 /wp-admin/plugins.php(192): plugin_sandbox_scrape('bs-swiper-main/...') #2 {main} thrown in /wp-content/plugins/bs-swiper-main/main.php on line 21

Hmm, usually there is no issue with automatic updates. However, please download https://github.com/bootscore/bs-swiper/releases/download/v5.3.1/bs-swiper-main.zip file directly and update plugin manually in backend.

Thanks for reporting. https://github.com/bootscore/bs-swiper/pull/47 will fix that.


Closing "Window" and title collision?

Hey,
So, this is might not be this libary's problem, although, it might need some additional functions to expose more things, such as a func (p *Panel) IsClosed().
But the owner of the other repo doesn't seem all that active, so I thought I post here too as you are more active and could possible help me out.
So, what say you about this: https://github.com/vurtun/nuklear/issues/361 ?
Thanks!

Hi, no quick answer on this from me, I'm quite busy too, also don't know Nuklear so deep as you in the referenced issue. I hope vurtun or dumblob will help sooner :)

Thanks for quick answer. After looking into it a bit more, it seems that this:
if nk.NkGroupBegin(ctx, s(n.Name), nk.WindowMovable|nk.WindowClosable|nk.WindowNoScrollbar|nk.WindowBorder|nk.WindowTitle) != 0 {...}
should actually make it so I don't enter when closing the "node" (panel), but instead I get:
nuklear.h:17834: nk_panel_layout: Assertion `!(layout->flags & NK_WINDOW_HIDDEN)' failed.
I'm just not sure exactly how this is suppose to work and be handled...

Unrelated: you can omit s() helper with the lates version of Nk, strings there are safe as for  now.

Ah. Cool.
Basically, this is about being able to deal with closed panels, and not windows. So, if you have suggestions here, please do share.
Thanks!

Maximum I can do here is to bump the header version :)

Hmm, isn't there a way to link the nuklear.h file so it's taken from the other repo when updating this repo? Although, not as "stable" then I suppose.

not stable
not go-gettable

Yeah, makes sense.


The operation couldn’t be completed. (OAuthSwiftError error -10.)

I received the error in the screenshot below when trying to connect my Twitter account to NetNewsWire on MacOS.

I already had Twitter connected to NNW on my iPad. After receiving this error, I disconnected Twitter from NNW on my iPad but was still unable to connect via my Mac. I was able to reconnected Twitter to NNW on my iPad afterward.

I'm embarrassed to say this issue was settled with a restart of my Mac.


we have an include problem - socket and file implementations conflict

I was writing a webserver code, it worked fine till I moved my function that called "write()" to another file. Suddenly it wasn't doing anything anymore, just giving errors.
I was attempting to see if I was getting the same write functions so I printed out the pointer:
printf("response.write=%x\n", (int) write);
turns out, this statement can't compile as write in "this file" is a macro!
Same problem exists with fcntl() function as nonblocking suddenly didn't work on stdio,
because all the levels of macros included the socket functions instead of the file functions.
CC /home/knoppix/GIT/esp-lisp-other/lisp.c
In file included from ../esp-open-rtos/libc/xtensa-lx106-elf/include/sys/fcntl.h:3:0,
from ../esp-open-rtos/libc/xtensa-lx106-elf/include/fcntl.h:1,
from /home/knoppix/GIT/esp-lisp-other/lisp.c:80:
../esp-open-rtos/libc/xtensa-lx106-elf/include/sys/_default_fcntl.h:44:0: error: "O_NONBLOCK" redefined [-Werror]
#define O_NONBLOCK _FNONBLOCK
^
In file included from ../esp-open-rtos/lwip/lwip/src/include/posix/sys/socket.h:33:0,
from /home/knoppix/GIT/esp-lisp-other/lisp.c:74:
../esp-open-rtos/lwip/lwip/src/include/lwip/sockets.h:278:0: note: this is the location of the previous definition
#define O_NONBLOCK  1 /* nonblocking I/O */
./libc/xtensa-lx106-elf/include/sys/_default_fcntl.h:23:#define _FNONBLOCK      0x4000  /* non blocking I/O (POSIX style) /
./libc/xtensa-lx106-elf/include/sys/_default_fcntl.h:24:#define _FNDELAY        _FNONBLOCK      / non blocking I/O (4.2 style) /
./libc/xtensa-lx106-elf/include/sys/_default_fcntl.h:44:#define O_NONBLOCK      _FNONBLOCK
./libc/xtensa-lx106-elf/include/sys/_default_fcntl.h:77:#define FNONBIO         _FNONBLOCK      / XXX fix to be NONBLOCK everywhere */
I'm not sure how to resolve this, but having macros overriding functions gets somewhat complicated..., also having two non-compatible implementations of functions get even worse...
As I can't use "files/io" and "sockets" in the same file.

Hi yesco,
Sorry for the delay in getting back to you. This is indeed an issue, both LWIP and newlib (libc) provide a "POSIX sockets" interface.
The long-term solution is to have newlib's system call implementations detect when a file descriptor belongs to LWIP and redirect those calls to use LWIP, while other operations use the serial port (or a filesystem if/when we support one.) This kind of falls under issue #55.
The short-term solution is to edit lwip/include/lwipopts.h (or make a copy in your local include directory, and edit that) then add this line:
#define LWIP_POSIX_SOCKETS_IO_NAMES     0

Now, you can use lwip_read(), lwip_write(), lwip_close(), lwip_fcntl() to refer to sockets and the normal names to refer to stdin/stdout.
You can see how this gets interpreted, here:
https://github.com/SuperHouse/esp-lwip/blob/c4765dccec5c01faeb87e5ed65db1c99ff5c9a05/src/include/lwip/sockets.h#L364
Angus