phi3 doesn't seem to accept SYSTEM prompt

What is the issue?
Regardless of what's in the modelfile, it seems phi3 doesn't take in the SYSTEM prompt at all. I've looked around and can't find anyone else discussing this. Assuming this is a bug of some kind.
OS
macOS
GPU
Apple
CPU
Apple
Ollama version
0.1.32

Took a cursory glance at the technical report^1 and I'm new to AI in general but it looks to me like Phi3 wasn't trained with a system prompt in mind, the template in the report shows only user and assistant.
<|user|>\n Question <|end|>\n <|assistant|>

The Modelfile^2 sample provided by Microsoft also doesn't include a system prompt. Perhaps @jmorganca or someone else from the Ollama team can shed some light on it. They know more than I do. 👍

Circling back to this after some testing. From my testing, I get that it kinda supports it but not really. The results are not consistent at all. I've tried different system prompts and some it kinds of adhered to and others ignore completely.
The best result I had was Ollama's Mario example where Phi3 was not acting as Mario, like for instance, Llama3 does. But instead acted as an assistant answering questions about the Mushroom Kingdom.
The template on the HuggingFace READMEs shows <|system|>, but not on the GGUF and ONNX variants. And the sample Modelfile on HuggingFace^1 again doesn't have <|system|>, so maybe it was trained with it for a bit and then they decided to scrap it?
If I was going to use Phi3 I wouldn't bother with a system prompt given these results.

@thinkverse - Thanks for your comments. Perhaps they did scrap the system prompt to keep the model's training simpler..? In any case, feeding in a system prompt as the user is good enough for a model this small, I guess.

@rb81 Phi 3 has been updated with a system prompt as per their published tokenizer configuration.
% ollama run phi3
>>> /set system You are a helpful assistant that always answers in french.
Set system message.
>>> Hi!
Bonjour ! En tant qu'assistant, je suis toujours prêt à répondre et à vous assister dans votre langue maternelle française.
Comment puis-je vous aider aujourd'hui ?

So basically the system prompts I was testing with were just bad. 😂 Time to learn how to write better prompts. 👍

@rb81 Phi 3 has been updated with a system prompt as per their published tokenizer configuration.
% ollama run phi3
>>> /set system You are a helpful assistant that always answers in french.
Set system message.
>>> Hi!
Bonjour ! En tant qu'assistant, je suis toujours prêt à répondre et à vous assister dans votre langue maternelle française.
Comment puis-je vous aider aujourd'hui ?

So, it seems setting the system prompt in runtime as you did works, but setting it in the modelfile doesn't. Have you had the same experience?

Does anyone involved in this issue understand how Phi-3-Mini's system prompt is handled under the hood of ollama? I'd like to replicate it in my own code, but I've run into the same issue that thinkverse spoke with regards to earlier (quoted below for clarity).

Took a cursory glance at the technical report1 and I'm new to AI in general but it looks to me like Phi3 wasn't trained with a system prompt in mind, the template in the report shows only user and assistant.
<|user|>\n Question <|end|>\n <|assistant|>

The Modelfile2 sample provided by Microsoft also doesn't include a system prompt. Perhaps @jmorganca or someone else from the Ollama team can shed some light on it. They know more than I do. 👍
Footnotes
1. [arxiv.org/html/2404.14219v1#S2](https://arxiv.org/html/2404.14219v1#S2) [↩](#user-content-fnref-1-07bb2b45304e3802ea204f5642c73b0b)

2. [Azure-Samples/Phi-3MiniSamples@`main`/ollama/Modelfile](https://github.com/Azure-Samples/Phi-3MiniSamples/blob/main/ollama/Modelfile?rgh-link-date=2024-04-23T20%3A15%3A02Z) [↩](#user-content-fnref-2-07bb2b45304e3802ea204f5642c73b0b)

I don't know,
adding this config to the tokenizer, it works
tokenizer.use_default_system_prompt = True
tokenizer.chat_template = "{{ bos_token }}{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + '<|end|>' }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + '<|end|>' }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n'  + message['content'] + '<|end|>' }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}"

it seems like Phi3 doesn't accept the system role, however, they said Phi3 has 3 roles in their cookbook:
https://github.com/microsoft/Phi-3CookBook/blob/main/md/02.QuickStart/Huggingface_QuickStart.md

i just noticed, there is no <|system|> token in thephi3's chat_template config. https://huggingface.co/microsoft/Phi-3-mini-4k-instruct/blob/main/tokenizer_config.json
adding this config to the tokenizer, it works
tokenizer.use_default_system_prompt = True
tokenizer.chat_template = "{{ bos_token }}{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + '<|end|>' }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + '<|end|>' }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n'  + message['content'] + '<|end|>' }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}"

I took chat_template config from https://huggingface.co/HuggingFaceH4/zephyr-7b-beta/blob/main/tokenizer_config.json

great, thank you! And you don't even need this line tokenizer.use_default_system_prompt = True

Thank you @CaptXiong , for pointing it out

@wuriyanto48 @CaptXiong - I'm assuming what you guys figured out requires modifying the source code and recompiling, correct? If so, can we get the project team to look at this?

@rb81 You don't need to recompile the source code, just like this
model_id = "microsoft/Phi-3-mini-4k-instruct"
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    trust_remote_code=True,
)

tokenizer = AutoTokenizer.from_pretrained(model_id)

messages = [
    {
        "role": "system",
        "content": "You are GoodBot, You are a helpful assistant.",
    },
    {
        "role": "user",
        "content": "who are you?"
    }

 ]

tokenizer.use_default_system_prompt = True
tokenizer.chat_template = "{{ bos_token }}{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + '<|end|>' }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + '<|end|>' }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n'  + message['content'] + '<|end|>' }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}"

pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    device=device,
)

generation_args = {
    "max_new_tokens": 500,
    "return_full_text": False,
    "temperature": 0.0,
    "do_sample": False,
}

output = pipe(messages, **generation_args)

hope this help

Hey folks, will close this for now, but let me know if you're still seeing issues.

Using python code on the Phi models I can get the phy models to follow a system instruction and only return what I instruct it to.
This does not work when the model is loaded into Ollama: See the examples below where I request that it only returns yes, no or I don't know.
I have attempted to include this in the user message but it doesn't seem to listen to that either. Is there anyway to get phi to listen to system messages in ollama.
Example in  Ollama:
Same behaviour whatever phi model I choose.
`$ ollama run phi3:3.8b-instruct
>>> /set system As a knowledge responder, your task is to provide a direct answer to a factual question. Your response options are limited to "Yes", "No", or "I dont know"
Set system message.
>>> /show system
As a knowledge responder, your task is to provide a direct answer to a factual question. Your response options are limited to "Yes", "No", or "I dont know"
>>> Does a cheetah have stripes?
No. A cheetah has spots instead of stripes, which help with camounerage in their natural habitat. The black tear marks
under the eyes may also aid them to see better at high speeds by counteracting sun glare off dusty surfaces. While they
are similar looking animals called leopards and jaguars that have striking orange or spotted coats, cheetahs do not have
stripes on their fur.
`
Example Test Script in python :
`import torch
from transformers import pipeline
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "microsoft/Phi-3-mini-4k-instruct"
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype="auto",
trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
messages = [
{
"role": "system",
"content": """As a knowledge responder, your task is to provide a direct answer to a factual question. Your response options are limited to "Yes", "No", or "I dont know"""
},
{
"role": "user",
"content": "Does a cheetah have stripes?",
}
]
tokenizer.chat_template = "{{ bos_token }}{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + '<|end|>' }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + '<|end|>' }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n'  + message['content'] + '<|end|>' }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}"
pipe = pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
device="cuda",
)
generation_args = {
"max_new_tokens": 500,
"return_full_text": False,
"temperature": 0.0,
"do_sample": False,
}
output = pipe(messages, **generation_args)
print(output)
`
Result: [{'generated_text': ' No'}]

phi3:3.8b-instruct is the q4_0 quantized version of the model.  If you use the fp16 version, it follows instructions better.
$ ollama run phi3:3.8b-mini-128k-instruct-fp16
>>> /set system As a knowledge responder, your task is to provide a direct answer to a factual question. Your response options are limited to "Yes", "No", or "I dont know"
Set system message.
>>> Does a cheetah have stripes?
No


feat: Receive the order's name from the frontend instead of matching order_id to name in backend

As per our discussion, since we want to map the order to its discussion in our backend, sending the order_id stays.
Can we close this issue ?


Turn on tree-sitter by default

Summary
Tree-sitter has been in testing for a while now. We're switching it on! This might affect how the syntax highlighting of some files looks, but nothing should look broken.
Motivation
Tree-sitter is a new incremental parsing system for Atom. It maintains a living parse tree, enabling all kinds of interesting extensions: search-for-similar-expression, refactoring assistance, and so on. None of that's implemented yet. For now, it's just making syntax highlighting blazing fast, and gives us the ability to embed languages within each other, perfectly highlighting Javascript inside <script> tags, HTML template strings in Javascript, <script> tags inside HTML template strings inside Javascript inside ERB, and on and on, until you throw away this whole artisanal templating system you've evidently grown and switch to React. Which we also parse.
Describe alternatives you've considered
We did consider not doing this, but decided instead to do it.
TODO

[ ] Test tree-stter with popular themes #17811
[ ] Update the flight manual

Good day,
I am of the opinion that the designers of this fine software does not use Atom.
Please; before making Tree-Sitter a default, look into making the standard snippets available again. The original grammar and Tree-Sitter based grammar do not share the scope name.
Other service providers (community packages) based on grammar scope names are also get broken. My humble suggestion is to at least have the grammar change mentioned in the flight-manual. Else have the depreciation-cop pick the fact up if a non-tree-sitter grammar is referenced.
Please consider making these prelude preparations before switching Tree-Sitter functionality on by default. Thanks in advance for your attention to this matter.
Regards from a mostly happy Atom user.

@danPadric Snippets have been available when using tree-sitter since Atom 1.25. See https://github.com/atom/atom/issues/16621.

@danPadric For keybindings I opened https://github.com/atom/atom/issues/17820. Is there any other issues you were considering? Please open a new issue for these!

I am of the opinion that the designers of this fine software does not use Atom.

@danPadric your comment above is a violation of the Atom Code of Conduct as it is insulting or derogatory. You may consider this an official warning.

@danPadric Which snippets aren't working for you? I know snippets didn't work when we first shipped Tree-sitter mode, but I thought that was fixed by https://github.com/atom/snippets/pull/264. Can you open an issue with detailed information about the problem you're seeing?

My humble suggestion is to at least have the grammar change mentioned in the flight-manual.

We do plan to do that.

Community Tree-sitter grammars seem to have a hard time installing; I tried on a new Ubuntu VM, and it couldn't install language-latex2e because of a missing make command.
I doubt Atom can package it's own build tools, but maybe it can catch this error and ask the user if they want to install it? It seems make is the only requirement.
This is particularly important as more grammars are made for Tree-sitter; many people will not have make installed, especially non-developers.

Dear Atom Contributors,
@lee-dohm:
You are right. My behaving was that of a child: emotionally charged, unsubstantiated and unprofessional. I have done wrong and regret my approach. I apologize to you and your fellow team members.
@queerviolet:
I am sorry for starting my conversation in this disrespectful way and on your topic. I am sorry.
@Ben3eeE:
After reading your reply (thank you), I realised the error must be mine mine. I followed up on your note. The short version - I have a localized problem on my machine (me := shame). That that means there is no reason for panic. My notes follows.
To all: I offer no excuses. My behaviour has been wrong. Please forgive me.

@danPadric Can you share the list of community packages you have installed? You can get this by running apm list --installed. It seems strange that snippets will stop working with tree-sitter 🙇

@danPadric Apology accepted. And thank you for working with us to help make Atom better :grinning:

@lee-dohm:
Thank you.

@Ben3eeE:
First note I have now 3 versions of Atom on my machine. 1 stable and 2 betas. One Beta has no add on packages. The other Beta has add-on packages that has been added/removed/updated since version 1.16 from a Windows 7 machine.
Please find below descriptions of one stable version and of one Beta version. The Beta version mentioned now is the one that has add-ons. THIS Beta version exhibits the same symptoms as with the stable version.
Is there a developer console command I could execute to list the installed and disabled packages for the Beta version? apm list does not work, even if running locally.
Atom V1.29.0: (1) installed and (2) disabled
C:\>apm list --installed
Community Packages (25) C:\Users\wbirm\.atom\packages
├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>(disabled)
├──<EMAIL_ADDRESS>(disabled)
├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>(disabled)
├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>(disabled)
├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>(disabled)
├──<EMAIL_ADDRESS>(disabled)
├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>(disabled)
├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>(disabled)
├──<EMAIL_ADDRESS>(disabled)
└──<EMAIL_ADDRESS>
C:\>apm list --disabled
Built-in Atom Packages (7)
├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>└──<EMAIL_ADDRESS>
Community Packages (9) C:\Users\dp\.atom\packages
├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>└──<EMAIL_ADDRESS>

Atom V1.30-Beta 1 with add-ons.
Directory of D:\AtomBeta\.atom\packages

2018/08/06  11:54    <DIR>          .
2018/08/06  11:54    <DIR>          ..
2017/11/29  11:38    <DIR>          autoclose-html
2018/08/06  11:53    <DIR>          autocomplete-python
2017/11/29  11:38    <DIR>          busy-signal
2018/08/02  20:08    <DIR>          duotone-dark-sea-syntax
2018/08/02  20:04    <DIR>          duotone-dark-space-syntax
2018/07/05  18:03    <DIR>          edit-in-new-tab
2018/06/28  08:24    <DIR>          fonts
2017/11/29  11:38    <DIR>          indent-guide-improved
2018/04/30  14:56    <DIR>          language-babel
2018/06/28  08:26    <DIR>          language-markdown
2018/05/26  18:09    <DIR>          language-scl
2018/05/26  18:09    <DIR>          language-stl
2017/11/29  11:38    <DIR>          MagicPython
2017/09/05  19:25    <DIR>          now-count-words (custom)
2018/04/21  19:01    <DIR>          one-coral-light-syntax
2018/08/06  11:45    <DIR>          one-dark-matte-syntax
2018/08/06  11:54    <DIR>          process-palette
2018/04/16  19:48    <DIR>          python-indent
2018/01/24  09:17    <DIR>          sequential-number
2018/01/29  21:45    <DIR>          typewriter
2018/01/11  16:01    <DIR>          wombat-dark-syntax
2017/11/29  11:38    <DIR>          Zen

Thank you for your attention.

@Ben3eeE:
The root cause was found.
In the list there are two grammar packages that are modified and customized versions that you will not find on public spaces (ie. Github). One of these contain the code lines:
# D:\AtomBeta\.atom\packages\language-stl\snippets\language-stl.cson
'*':
  'Copyright Notice':
    'prefix': 'legal'
    'body': '// Copyright (c) ${1:2016} ${2:Copyright Holder} ${3:All Rights Reserved.}$4'

Amazingly the '*' key is no problem for for non-tree-sitter grammars. This does however override all snippets for all tree-sitter grammars!   😮 😊

@danPadric Thank you :bowing_man: I can reproduce this behavior.
@maxbrunsfeld @queerviolet This seems like something we should investigate before turning tree-sitter on by default.

@danPadric The issue you identified is fixed on master. Thanks!

@Ben3eeE ,  @maxbrunsfeld:
Thank you for your dedication to the topic.
I understand that the * key in the global user snippets.cson identify snippets that are applied to all grammar scopes. May I assume that this concept will be available for Tree-Sitter and Textmate grammars alike?
The reason for asking: The newly integrated changes is labelled as legacy. Now I am left to wonder if the apply-to-all-grammars (* key) would still work for Tree-Sitter grammars as intended.

* English is not my 1st language: No disrespect intended with the above text.

May I assume that this concept will be available for Tree-Sitter and Textmate grammars alike?

Yes.

Other service providers (community packages) based on grammar scope names are also get broken.

I believe this is a real problem that needs to be fixed before automatically changing the default behavior.
A couple examples I know of:
https://github.com/vn-ki/bracket-colorizer/pull/10
https://github.com/AsaAyers/js-hyperclick/issues/95

@UziTech This is fixed on master: https://github.com/atom/atom/pull/17881.

@maxbrunsfeld how does that fix the difference in scopes for elements?
e.g. js regex scopes:
without tree-sitter:

source.js
string.regexp.js

with tree-sitter:

javascript
program
expression_statement
assignment_expression
regex
regex_pattern

Packages that use those scope names are going to need to be updated to work with tree-sitter like bracket-colorizer

I’m confused; did you rebuild master after the PR I linked to was merged? The scopes aren’t like that any more; that’s why I’m saying the problem is addressed.
There may still be some slight differences, but it’ll just be a matter of tweaking the scope mappings in the various tree-sitter grammar files.


unhelpful error message when using a class template which doesn't satisfy the expression concept

The following definition of an expression template attempts to use std::tuple as the element container.
    template <expr_kind Kind, typename T>
    struct m_expr {
        constexpr static expr_kind kind = Kind;

        using tuple_type = std::tuple<T>;

        tuple_type elements;
    };

    int main()
    {
        constexpr auto e = make_terminal<m_expr>(1);
    }

It fails to compile with a unhelpful error message error: static_assert failed "evaluate() is only defined for expressions."
I propose the following version of is_expr<> as a remedy.
    template<
        typename Expr,
        typename = detail::void_t<>,
        typename = detail::void_t<>>
    struct is_expr : std::false_type
    {
    };

    template<typename Expr>
    struct is_expr<
        Expr,
        detail::void_t<decltype(detail::remove_cv_ref_t<Expr>::kind)>,
        detail::void_t<decltype(std::declval<Expr>().elements)>>
        : std::true_type
    {
        static_assert(
            std::is_same<
                std::remove_cv_t<decltype(detail::remove_cv_ref_t<Expr>::kind)>,
                expr_kind
            >::value,
            "expression template needs a yap::expr_kind tag");
        static_assert(
            hana::is_a<
                hana::tuple_tag,
                decltype(std::declval<Expr>().elements)
            >(),
        "expression template needs a hana tuple like container for elements");
    };

Maybe the fall-back instantiation of is_expr should also static_assert?

This is best we can do in this case, afaict.  We cannot add the static_asserts) above without breaking SFINAE uses of is_expr<>, which is one of its major use cases.  If you have another way to report errors more gracefully without resorting to static_assert(), I'd be happy to use it.

They do.  usually it's just static_assert(is_expr::value, "Expr must be a YAP expressions") or similar.  It's then up to the user to figure out why Expr is not a model of Expression.  The requirements are extremely simple for the user to check/audit/enforce.


Please export extensionAPIClient

Is your feature request related to a problem? Please describe.
Occasionally would like to write a small Lambda extension to do some processing after my main Lambda handler has returned a response. While the requirements for implementing an extension are well-documented, there's no official 1st-party public Go package to help with implementation.
Describe the solution you'd like
package lambda already has implemented an extension client, but it is unexported. I'd like to be able to create a new extension client by calling some constructor NewExtensionAPIClient (which could be in package lambda or in some other package), register event types I'm interested in, and define an extension event handler.
It seems to me that this would not add much maintenance burden since the extension client is already implemented for the sake of the WithEnableSIGTERM option. The difference here would be supporting the exported API for the extension client.
Describe alternatives you've considered
I could implement and maintain an extension API client myself; but the same is true for implementing a runtime API client. I'd rather rely on an officially supported implementation.
I could find a 3rd-party implementation, but since there's no single or few 3rd-party implementations that have gained community consensus, I don't have any assurances about the quality of the implementation.

I'd like to express my thanks to those maintaining this repository. I use it a lot, and I appreciate your work.

Sure :)
The client isn't quite feature complete, so I'll want to take a chance myself to 'finish' it, and mull over some cosmetic changes to naming, package structure, etc.


Nvidia Shield only 480p quality

The Nvidia Shield has full Netflix support but somehow I get horrible quality through this plugin in Kodi.. I asked on the Kodi forum and searched here and online for answers, but sadly no help.. I've entered the full ESN number in the addon settings, enabled HEVC, set InputStream Adaptive to manual to choose the quality while playing and set minimum bandwith to 500000000.. It's only giving me 480p options.. Does anyone have any idea what could be causing this? I'm running version 0.13.21 on Kodi 18 RC5..

I can succesful run full hd. Just gonto InputStream Adaptive add-on settings, turn on Override HDCP ststus. Then you get all movie full HD. You can see video info during playback, netflix add on is using h.264. This old video codec, and the quality is not same as netflix app.
To fix this, just go to netflix add on settings, turn on Enable VP9 profiles. This is new video codec that will improved video quality and use less 40% data bandwith. You also cab turn on HEVC profiles for 4k video. HEVC is latest codec, superior quality than VP9.
No need to delete anything, just stop video and play again. Now you get netflix add on same video quality as netflix app.

There was a bug in inpustream.adaptive where if you selected 720p, you would get a lower quality stream. And if you selected 1080p you would get the 720p stream. It is fixed in peak3d/inputstream.adaptive#259


Clean mode1/mode2 / primary/secondary handling

Done:

Proper flag handling
Communication to right remote
Storing


add base Bucket contract

Add a base Bucket contract from which GiftBucket and NFTBucket inherit.

@bitgamma since a lot of things moved around I think it would be easier to review the final contract files instead of the changes.


Ignore AuthFailed errror while listing properties of parent directory

Transfer to a filestore with a read-only SAS fails because we try
listing properties for parent directory, and fail with permission mismatch.
With this change, we ignore the error and try creating parent directories.
Also, we try matching the source and destination file lengths in
epilogue. The error here will be selectively ignored as well.

BTW, @nakulkar-msft , for future PRs, please don't use numerical branch names (like b6974627).  Please name the branch with a meaningful name, and start the name with fix/ if its a bug fix and feature/ if its a new feature.

  		_, err := curDirURL.Create(ctx, azfile.Metadata{}, azfile.SMBProperties{})

This is an old TODO. Feel free to remove it.

Refers to: ste/sender-azureFile.go:443 in 126ceaf. [](commit_id = 126ceafce2e2c6ed4506880936c2160ad19e174f, deletion_comment = False)


Add integration tests for trial-based experiments

Find precursor projects with trialized CaImg experiments
extend workflow-calcium-imaging to include trialized example
corresponding integration test

Successfully added trialization. Still lacking integration tests


Connection Leak

(Maybe related to https://github.com/redisson/redisson/issues/3840)
Expected behavior
Following fresh startup of service utilizing Redisson client, AWS ElastiCache nodes each maintain consistent ~1k active connections from our service without change (with exception of node failovers, re-sharding, etc)
Actual behavior
Slow connection leak after deployment of 3.16.3 client & constant spam of new connections being created. Previous client versions (tested all of 3.15.6, 3.16.0, 3.16.1, 3.16.2) have maintained constant ~1k connections per node without new connection spam.
Pattern of connection leak is unclear - connections increased continuously for ~11 hours, slowly decreased over another ~10 hours. Starting experiencing complete cache failure a bit before lowest peak of active connections during leak was hit.
AWS ElastiCache metrics:
(First drop is from trying re-deployment, where connection leak occurred again)

Note that the read replicas of shard 0003 (0003-001, 0003-003) are the problematic ones in this case.
Nodes (as reported by ElastiCache IsMaster metric)
Shard 0001:

Node 001: Master
Node 002: Read-Replica

Shard 0002:

Node 001: Master
Node 002: Read-Replica

Shard 0003:

Node 001: Read-Replica
Node 002: Master
Node 003: Read-Replica

Sample of debug-level logs (sanitized): https://pastebin.com/raw/RKJaYeX7
Reverting to previous client version resolves issue.
Steps to reproduce or test case
We have been able to repeat this issue across service deployments, but can't give general reproduction steps.
Redis version
5.0.6 (Engine: Clustered Redis)
Redisson version
3.16.3
Redisson configuration
Transit Encryption: Enabled
Timeout: 200 ms
Connect Timeout: 20 s
Check Slots Coverage: false
Retry Attempts: 1
Retry Interval: 500 ms

Can you try attached version?
redisson-3.16.4-SNAPSHOT.jar.zip

Could you push the source code to a branch or provide some type of diff check from 3.16.3? For security & other reasons I cannot pull in that jar file into our environment for usage but I can manually patch source code changes between official Redisson client releases to test.

@afuelcell
You can build master branch. I think it should be fixed by https://github.com/redisson/redisson/commit/33e5b5872b098c13cf1bbf70f8152f7f3d94dc7c

According to logs it's related to ClusterConnectionManager.
11 Oct 2021 15:23:58,035 [INFO]  (redisson-netty-11-7) org.redisson.cluster.ClusterConnectionManager: slave: rediss://<priv-ip-5>:6379 added for slot ranges: [[12678-14002], [2570-5300], [7572-8936], [11272-11311]]

It detects ip changes for slave host name.

Searching logs in past hour for "added for slot ranges":
https://pastebin.com/raw/SKfvYkzV

Seems that the problem is in hostname resolution. Can you switch to netty 4.1.67.Final ?

Building in netty 4.1.67.Final seems to have partially worked. The leak appeared to be prevalent from the start of the deployment of our service and was ongoing up until our last host deployed, at which the connection counts returned to their expected value.

Start & end of the "leak" align with hosts for service coming up.
During the deployment, I had checked and noticed the same "added for slot ranges" logs were appearing that are now no longer appearing after all of the hosts have fully deployed.

It must've been some stroke of luck, because after another try of service deployment we see the same patterns, logs as before (with the slot ranges logs) where the leak has not resolved after deployment.

@afuelcell
I need to sort out what is the problem. Can you set TRACE logging level for io.netty.resolver package and share output with this issue?

@afuelcell
I need to sort out what is the problem. Can you set TRACE logging level for io.netty.resolver package and share output with this issue?

Here is how the logs look for the most part from what I can see with the connection leak & netty trace logging enabled -
https://pastebin.com/raw/V5QjqefH
Will test out the other rewrite you gave when I get the chance (likely tomorrow).

Can you try this version of resolveIP method (logs added) ?
    private static final Map<RedisURI, RedisURI> MAPPING = new LRUCacheMap<>(100, 0, TimeUnit.DAYS.toMillis(30));

    protected RFuture<RedisURI> resolveIP(String scheme, RedisURI address) {
        if (address.isIP()) {
            RedisURI addr = applyNatMap(address);
            return RedissonPromise.newSucceededFuture(addr);
        }

        RPromise<RedisURI> result = new RedissonPromise<>();
        AddressResolver<InetSocketAddress> resolver = resolverGroup.getResolver(getGroup().next());
        InetSocketAddress addr = InetSocketAddress.createUnresolved(address.getHost(), address.getPort());
        Future<List<InetSocketAddress>> future = resolver.resolveAll(addr);
        future.addListener((FutureListener<List<InetSocketAddress>>) f -> {
            if (!f.isSuccess()) {
                log.error("Unable to resolve " + address, f.cause());
                result.tryFailure(f.cause());
                return;
            }

            List<InetSocketAddress> addrs = f.getNow();
            RedisURI currentURI = MAPPING.get(address);
            if (currentURI != null) {
                InetSocketAddress prevAddr = InetSocketAddress.createUnresolved(currentURI.getHost(), currentURI.getPort());
                log.info("resolved addrs: {}, currentAddr: {}, in list:{}", addrs, prevAddr, addrs.contains(prevAddr));
                if (addrs.contains(prevAddr)) {
                    result.trySuccess(currentURI);
                    return;
                }
            }

            InetSocketAddress s = addrs.get(0);
            RedisURI uri = new RedisURI(scheme + "://" + s.getAddress().getHostAddress() + ":" + address.getPort());
            uri = applyNatMap(uri);
            MAPPING.put(address, uri);
            log.info("added addrs: {}, currentAddr: {}, currentURI: {}", addrs, s, currentURI);
            result.trySuccess(uri);
        });
        return result;
    }

Updated version of the method.
    private static final Map<RedisURI, RedisURI> MAPPING = new LRUCacheMap<>(100, 0, TimeUnit.DAYS.toMillis(30));

    protected RFuture<RedisURI> resolveIP(String scheme, RedisURI address) {
        if (address.isIP()) {
            return RedissonPromise.newSucceededFuture(address);
        }

        RPromise<RedisURI> result = new RedissonPromise<>();
        AddressResolver<InetSocketAddress> resolver = resolverGroup.getResolver(getGroup().next());
        InetSocketAddress addr = InetSocketAddress.createUnresolved(address.getHost(), address.getPort());
        Future<List<InetSocketAddress>> future = resolver.resolveAll(addr);
        future.addListener((FutureListener<List<InetSocketAddress>>) f -> {
            if (!f.isSuccess()) {
                log.error("Unable to resolve " + address, f.cause());
                result.tryFailure(f.cause());
                return;
            }

            List<InetSocketAddress> addrs = f.getNow();
            RedisURI currentURI = MAPPING.get(address);
            if (currentURI != null) {
                byte[] ipaddr = NetUtil.createByteArrayFromIpAddressString(currentURI.getHost());
                InetSocketAddress prevAddr = new InetSocketAddress(InetAddress.getByAddress(address.getHost(), ipaddr), currentURI.getPort());
                log.info("resolved addrs: {}, currentAddr: {}, in list:{}", addrs, prevAddr, addrs.contains(prevAddr));
                if (addrs.contains(prevAddr)) {
                    result.trySuccess(currentURI);
                    return;
                }
            }

            InetSocketAddress s = addrs.get(0);
            RedisURI uri = new RedisURI(scheme + "://" + s.getAddress().getHostAddress() + ":" + address.getPort());
            uri = applyNatMap(uri);
            MAPPING.put(address, uri);
            log.info("added addrs: {}, currentAddr: {}, currentURI: {}", addrs, s, currentURI);
            result.trySuccess(uri);
        });
        return result;
    }

@afuelcell
I modified three classes. Please add them and try to reproduce the issue.
[archive updated] redisson.zip

This seems to have fixed the issue as far as I can tell. Cannot reproduce the issue on either of the two fleets that I was seeing the leak on before. The current connection counts are stable upon deployment and there are no constant new connection spikes as we had seen previously.
Here are some sample redis logs for verification purposes if needed: https://pastebin.com/raw/whFXGjDB

I am seeing different patterns for the failover (no current connection increase to the new primary) compared to previous version

can you share logs with recorded failover?

After the time of starting node failover, we have seen increased latency to our service that hasn't recovered. Here is logs briefly after failover, and before did the addition of read-replica: https://pastebin.com/raw/w0u95Efb

You may turn off logs for io.netty package. Cluster node state doesn't report any failed node.


Support new transaction types coming from core

Chore
The new core features introduce new transaction types that we need to ensure are supported in the browser wallet ready for Deployment 1.

Successor Markets - new transaction type
Governance Transfers - new transaction types
Stop Orders - new transaction types
Iceberg Orders  - new transaction types

Tasks

[ ]
[ ]
[ ]

Notes

These will be automatically supported once the protos are in an official release 👍

Yep, I know that's the theory.... but this is a placeholder ticket to remind us that we will need to check and validate this when we get close to the release :)


Adding new members without district offices

I'm trying to add John Curtis, R-UT, to the legislators-current.yaml file, since he's being sworn in today. He doesn't appear to have any district offices yet. Without an entry (including an office address) in legislator-district-offices.yaml, the validate.py test fails.
I'm not sure what the best way to handle this is. It seems like we should be able to list new legislators as soon as they're sworn in, even if we don't know the address of a district office. One option might be to include them in the legislator-district-offices.yaml file, but without any offices (right now doing that also results in an error). Another might be to make missing offices a warning rather than an error.
Does anyone have thoughts about this? Or is there a way to handle it already and I'm just missing it?

Thanks for getting on this. What about looking at the swearing-in date. We can make it a warning if it's two months (?) after swearing-in without district offices?

Sounds good to me -- not ideal that whoever makes the first change after two months will see an error even if it's unrelated to their change. But we can encourage whoever adds the member in the first place to keep an eye out for district offices and add when available.
Here's a pull request with that change: https://github.com/unitedstates/congress-legislators/pull/524


removing retain cycles

I've did some profiling to my app that uses this lib, found couple of cycles and found un unowned closures that I've fixed
thanks

Wow, thank you for this! Feel bit bad for having so many issues in the code 😄

No worries, keep it up!


ValueError: The filepath provided must end in .keras (Keras model format). Received: filepath=Model_ckpt.h5

Issue Type
Support
Source
source
Keras Version
3.2.1
Custom Code
Yes
OS Platform and Distribution
Linux Ubuntu 22.04.4
Python version
3.11.5
GPU model and memory
16 gm
Current Behavior?
please solve this error
Standalone code to reproduce the issue or tutorial link
CKPT_path = "Model_ckpt.h5"
checkpointing_cb = tf.keras.callbacks.ModelCheckpoint(CKPT_path, save_best_only=True)

# Orginal train

EPOCHS = 30
VALIDATION_SET = (X_valid, y_valid)

history = model_clf.fit(X_train, y_train, epochs=EPOCHS,
                    validation_data=VALIDATION_SET, batch_size=32, callbacks=[tb_cb, early_stopping_cb,checkpointing_cb] )

Relevant log output
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[37], line 2
      1 CKPT_path = "Model_ckpt.h5"
----> 2 checkpointing_cb = tf.keras.callbacks.ModelCheckpoint(CKPT_path, save_best_only=True)

File ~/anaconda3/lib/python3.11/site-packages/keras/src/callbacks/model_checkpoint.py:191, in ModelCheckpoint.__init__(self, filepath, monitor, verbose, save_best_only, save_weights_only, mode, save_freq, initial_value_threshold)
    189 else:
    190     if not self.filepath.endswith(".keras"):
--> 191         raise ValueError(
    192             "The filepath provided must end in `.keras` "
    193             "(Keras model format). Received: "
    194             f"filepath={self.filepath}"
    195         )

ValueError: The filepath provided must end in `.keras` (Keras model format). Received: filepath=Model_ckpt.h5

I have the same problem in Kaggle notebook while executing the following code:
history=model1.fit(train_img_datagen,           steps_per_epoch=steps_per_epoch,           epochs=100,           verbose=1,           callbacks = [     keras.callbacks.ModelCheckpoint('UNet_100.h5', save_weights_only=False, save_best_only=True, monitor='val_loss', mode='min'),     keras.callbacks.ReduceLROnPlateau(), ],           validation_data=val_img_datagen,           validation_steps=val_steps_per_epoch,           )

@kishan042p I have just solved the problem by installing an older versio of keras:
!pip install keras==2.15.0
import keras
print(keras.version)
But you have to restart your kernel before using it,
Good luck

But i Want to save model in .h5 file not in .keras so please provide solution regarding it.

@kishan042p I have just solved the problem by installing an older versio of keras:
!pip install keras==2.15.0 import keras print(keras.version)
But you have to restart your kernel before using it,
Good luck

But i Want to save model in .h5 file not in .keras so please provide solution regarding it.

Yes I use my old code, where I save my model in .h5 file not .keras

In Keras 3, for checkpoint filepath, you need to provide in .keras format only, if you're saving only weights file name should end with .weights.h5.
You can save and load the model in .h5 using below method
https://keras.io/guides/migrating_to_keras_3/#saving-a-model-in-the-tf-savedmodel-format

In Keras 3, if you want to save your model in .h5 file, set "save_weights_only=True", and change your flie name to "xxx.weights.h5".
for your code:
checkpointing_cb = tf.keras.callbacks.ModelCheckpoint(CKPT_path, save_best_only=True, save_weights_only=True)

https://keras.io/guides/migrating_to_keras_3/#saving-a-model-in-the-tf-savedmodel-format
"How to fix it: use model.export(filepath) instead of model.save(filepath)"

In Keras 3, if you want to save your model in .h5 file in any case, set "save_weights_only=True", and change your flie name to "xxx.weights.h5".
for your code:
CKPT_path = "Model_ckpt.weights.h5"
checkpointing_cb = tf.keras.callbacks.ModelCheckpoint(CKPT_path, save_best_only=True, save_weights_only=True)


Paraswap - XYK pallet

Grant Application
This application is (select one):

[x] Speculative (use this by default)
[ ] an RFP response

This application is (select one):

[ ] Public (fully)
[x] Public with private finances

Abstract
Paraswap is a Substrate pallet that implements the constant product market maker model (XYK model) for automated market makers (AMM). Pallet will allow other parachains to include AMM features, that will provide decentralized exchange of tokens.
Exchange of assets is one of the fundamental operations in crypto ecosystems and provides liquidity for endless blockchain usecases.
We choose peer-to-contract exchanging as it proved popular in Ethereum DeFi ecosystem and it can serve a Price oracle service for the Polkadot ecosystem.
When demand will be validated, we will consider other AMM models in the future.
We will develop a pallet that implements XYK model to allow:

Internal Parachain swapping through Assets pallet
External swapping between two other parachains (through XCMP)

Checklist

[x] The grants document has been read and understood.
[ ] The Google Form will be completed accurately. Note that the Google Form requires the pull request URL.
[x] Abstract (above) is succinct and complete.
[x] The application is being included into the correct directory: either 'targeted' or 'speculative'.
[x] The application includes a project description.
[x] The application includes all names of team members.
[x] The application includes a description of the team's experience.
[x] The application includes all necessary links (e.g. GitHub and LinkedIn)
[ ] The "Development Roadmap" section in the application has a timeline of development ("milestones").
[ ] The "Development Roadmap" section in the application has an estimate of funds required.
[x] The "Development Roadmap" section gives an indication of the team's long term plans.
[ ] The "Development Roadmap" section includes documentation as a deliverable for at least one milestone.

Has this model received more analysis since the last Vitalik's post? Could you please provide/link some info on it's use cases and pitfalls. E.g. can it be used for prediction market instead of LMSR or it's limited to some special cases of trading?
I'm not from the grants team, just curious.


Fix GitLab merge request merge_status enum

Reference: https://gitlab.com/gitlab-org/gitlab/-/blob/77c8b68d9eec9624ca62fae0c12e52ab141aaf6a/app/models/merge_request.rb#L187-192

Fails

:no_entry_sign:

danger-python failed.

Log

  Generated by :no_entry_sign: dangerJS against 7352046eb8b1b012348f96773f286c6f164784da

👍🏻


The trait bound chrono::NaiveDateTime: diesel::Expression is not satisfied

Which versions of Rust and Diesel are you using?
Rust 1.18.0, Diesel 0.13.0.

Which feature flags are you using?

diesel = { version = "0.13", features = ["postgres", "chrono"] }
diesel_codegen = { version = "0.13", features = ["postgres"] }
chrono = { version = "0.4", features = ["serde"] }
serde = "1.0"
serde_derive = "1.0"
serde_json = "1.0"

What are you trying to accomplish?

Trying to upgrade from Diesel 0.11, Serde 0.9 and Chrono 0.3 to the above versions.

What is the full error you are seeing?

I'm seeing the following for each AsChangeset that uses chrono::NaiveDateTime:
error[E0277]: the trait bound `chrono::NaiveDateTime: diesel::Expression` is not satisfied
  --> src/models/emails.rs:37:10
   |
37 | #[derive(AsChangeset, Clone, Debug)]
   |          ^^^^^^^^^^^ the trait `diesel::Expression` is not implemented for `chrono::NaiveDateTime`
   |
   = note: required because of the requirements on the impl of `diesel::Expression` for `&chrono::NaiveDateTime`
   = note: required because of the requirements on the impl of `diesel::expression::AsExpression<diesel::types::Timestamp>` for `&chrono::NaiveDateTime`
   = note: this error originates in a macro outside of the current crate

How can we reproduce this?

I think this should be reproducible with the above versions of Chrono and Diesel by having a table that has a column of type chrono::NaiveDateTime, then having an AsChangeset struct that tries to update said NaiveDateTime column.
I'm using NaiveDateTime in Rust for Postgres' TIMESTAMP:
updated_at TIMESTAMP NOT NULL default now()

I've been going round in circles with this, but if it isn't actually a Diesel issue then I apologise in advance. Thanks for all the work on Diesel, it's excellent! :)

Very nice issue description! (We should steal this and make it a template!)
I think your problem stems from chrono recently yanking its 0.3.1 version and releasing 0.4 with serde 1.0 support instead (this was a breaking change in 0.3.1!). Diesel 0.13.0 does not support chrono 0.4. Which means two versions of chrono will be built, and they are note compatible to each other.
Can you try using diesel master instead?
diesel = { git = "https://github.com/diesel-rs/diesel", features = ["postgres", "chrono"] }

I've been going round in circles with this, but if it isn't actually a Diesel issue then I apologise in advance. Thanks for all the work on Diesel, it's excellent! :)

Thanks :) If it turns out this is a problem with two chrono versions, you can use cargo-tree to see this more easily.

Diesel 0.13.0 does not support chrono 0.4

Ah, that's probably what the problem is then! I saw this line and assumed it did.

Can you try using diesel master instead?

I wasn't expecting such a quick reply, I'll try it out as soon as I'm back at my work machine and let you know.

Very nice issue description! (We should steal this and make it a template!)

Thanks! I can't take credit for it though, I just filled in the details you ask for :)

Thanks! I can't take credit for it though, I just filled in the details you ask for :)

lol 😅

Right. So Diesel's master gets me past the original issue but I now have a different "this error originates in a macro outside of the current crate" error:
error[E0277]: the trait bound `diesel::pg::PgConnection: diesel::connection::Connection` is not satisfied
  --> src/database.rs:6:1
   |
6  | / lazy_static! {
7  | |     static ref CONNECTION: r2d2::Pool<ConnectionManager<PgConnection>> = {
8  | |         let config = r2d2::Config::default();
9  | |         let database_url = env::var("DATABASE_URL").expect("DATABASE_URL must be set");
...  |
12 | |     };
13 | | }
   | |_^ the trait `diesel::connection::Connection` is not implemented for `diesel::pg::PgConnection`
   |
   = note: required because of the requirements on the impl of `r2d2::ManageConnection` for `r2d2_diesel::ConnectionManager<diesel::pg::PgConnection>`
   = note: required by `r2d2::Pool`
   = note: this error originates in a macro outside of the current crate

Things I've tried:

Getting diesel_codegen from master too
Getting r2d2-diesel from master (though I wasn't expecting that to change anything since there hasn't been a commit to it since the last release)
Made sure I'm using the latest versions of r2d2 and r2d2-diesel.

database.rs:
use diesel::pg::PgConnection;
use r2d2;
use r2d2_diesel::ConnectionManager;
use std::env;

lazy_static! {
    static ref CONNECTION: r2d2::Pool<ConnectionManager<PgConnection>> = {
        let config = r2d2::Config::default();
        let database_url = env::var("DATABASE_URL").expect("DATABASE_URL must be set");
        let manager = ConnectionManager::<PgConnection>::new(database_url);
        r2d2::Pool::new(config, manager).expect("Failed to create pool.")
    };
}

pub fn connection() -> r2d2::Pool<ConnectionManager<PgConnection>> {
    CONNECTION.clone()
}

Feel free to close this when it gets too off topic from the original issue /  becomes too specific to just how I'm using it.

Does cargo tree give you two diesel versions? I'd assume so, because r2d2-diesel uses the latest diesel, not its master branch.
Maybe a better way to get this going is to not specify diesel master, but 0.13.0 and adding
[replace]
"diesel:0.13.0" = { git = "https://github.com/diesel-rs/diesel" }

This replace every occurrence of diesel 0.13 in your dependency graph. (You might need to add the same lines for tje other diesel-* crates.)

@killercup You're a gentleman and a scholar! Thanks so much for taking the time to help!

For anyone else who finds themselves here:
Add Diesel and co. as you would usually:
diesel = { version = "0.13.0", features = ["postgres", "chrono"] }
diesel_codegen = { version = "0.13.0", features = ["postgres"] }
r2d2 = "0.7"
r2d2-diesel = "0.13.0"

As @killercup said, replace the version of Diesel you're using like so:
[replace]
"diesel:0.13.0" = { git = "https://github.com/diesel-rs/diesel" }

Run cargo update to make sure r2d2-diesel is using the latest version of Diesel it can.
Build.

Thanks again @killercup 😄

Thank you! I'm glad this works for you. With the next release of Diesel, 0.14, you should only need to increment the diesel and r2d2 version, remove the [replace], and it should all just work™ :)

Just confirming that Diesel 0.14 does indeed resolve the above issues. Good work folks 👍

Thanks for confirming. :heart:

Actually. I use 0.14 and 0.14.1 now and I have these errors:
error[E0277]: the trait bound `chrono::NaiveDateTime: phone::_IMPL_DESERIALIZE_FOR_PhoneInfo::_serde::Serialize` is not satisfied
 --> src/models.rs:9:88
  |
9 | #[derive(Debug, Clone, Queryable, Identifiable, Insertable, Associations, AsChangeset, Serialize, Deserialize)]
  |                                                                                ^^^^^^^^^ the trait `phone::_IMPL_DESERIALIZE_FOR_PhoneInfo::_serde::Serialize` is not implemented for `chrono::NaiveDateTime`
  |
  = note: required by `phone::_IMPL_DESERIALIZE_FOR_PhoneInfo::_serde::ser::SerializeStruct::serialize_field`

error[E0277]: the trait bound `chrono::NaiveDateTime: phone::_IMPL_DESERIALIZE_FOR_PhoneInfo::_serde::Deserialize` is not satisfied
 --> src/models.rs:9:99
  |
9 | #[derive(Debug, Clone, Queryable, Identifiable, Insertable, Associations, AsChangeset, Serialize, Deserialize)]
  |                                                                                ^^^^^^^^^^^ the trait `phone::_IMPL_DESERIALIZE_FOR_PhoneInfo::_serde::Deserialize` is not implemented for `chrono::NaiveDateTime`
  |
  = note: required by `phone::_IMPL_DESERIALIZE_FOR_PhoneInfo::_serde::de::SeqVisitor::visit`

error[E0277]: the trait bound `chrono::NaiveDateTime: phone::_IMPL_DESERIALIZE_FOR_PhoneInfo::_serde::Deserialize` is not satisfied
 --> src/models.rs:9:99
  |
9 | #[derive(Debug, Clone, Queryable, Identifiable, Insertable, Associations, AsChangeset, Serialize, Deserialize)]
  |                                                                                ^^^^^^^^^^^ the trait `phone::_IMPL_DESERIALIZE_FOR_PhoneInfo::_serde::Deserialize` is not implemented for `chrono::NaiveDateTime`
  |
  = note: required by `phone::_IMPL_DESERIALIZE_FOR_PhoneInfo::_serde::de::MapVisitor::visit_value`

error: aborting due to 3 previous errors

error: Could not compile `omp-mdm-server`.

To learn more, run the command again with --verbose.

#[derive(Debug, Clone, Queryable, Identifiable, Insertable, Associations, AsChangeset, Serialize, Deserialize)]
#[table_name = "phones"]
#[primary_key(imei)]
#[belongs_to(User)]
pub struct Phone {
    pub imei: i64,
    pub model: String,
    pub vendor: String,
    pub manufacturer: String,
    pub last_sync: NaiveDateTime, // In UTC
    pub user_id: i32,
}

[dependencies]
rocket = "0.2.8"
rocket_codegen = "0.2.8"
rocket_contrib = "0.2.8"
serde = "0.9"
serde_derive = "0.9"
serde_json = "0.9"
r2d2 = "0.7.2"
r2d2-diesel = "0.14"
chrono = { version = "0.4", features = ["serde"] }

[dependencies.diesel]
version = "0.14"
default-features = false
features = ["sqlite", "chrono"]

[dependencies.diesel_codegen]
version = "0.14"
default-features = false
features = ["sqlite"]

@vityafx these look like serde issues—most likely because you are using serde 0.9 and chrono requires 1.0.

@killercup oh well, yes, totally forgot about that but I did knew that! :-D Just rocket uses old serde, this is the problem, I guess, but I can't change it :(

Rocket 0.3 will include it: https://github.com/SergioBenitez/Rocket/issues/273#issuecomment-301339550
Maybe you can use rocket's master branch until then?

@killercup a very good notice, I have just found this information too (about it's master branch), but thanks for such a nice help here anyway!

I have read this and I still do not understand the fix.
I am using diesel 0.16.0 and master  c5fa99574673e09e6e5029cda517802a82f66933
error[E0277]: the trait bound models::chrono::NaiveDateTime: diesel::types::FromSqlRow<diesel::types::Nullable<diesel::types::Timestamp>, diesel::pg::Pg> is not satisfied
--> src/main.rs:98:10
|
98 |         .get_result(conn)
|          ^^^^^^^^^^ the trait diesel::types::FromSqlRow<diesel::types::Nullable<diesel::types::Timestamp>, diesel::pg::Pg> is not implemented for models::chrono::NaiveDateTime
|
= help: the following implementations were found:
<models::chrono::NaiveDateTime as diesel::types::FromSqlRow<diesel::types::Timestamp, DB>>
<models::chrono::NaiveDateTime as diesel::types::FromSqlRow<diesel::types::Timestamptz, DB>>
= note: required because of the requirements on the impl of diesel::types::FromSqlRow<(diesel::types::Integer, diesel::types::Nullable<diesel::types::Timestamp>, diesel::types::Text), diesel::pg::Pg> for (i32, models::chrono::NaiveDateTime, std::string::String)
= note: required because of the requirements on the impl of diesel::Queryable<(diesel::types::Integer, diesel::types::Nullable<diesel::types::Timestamp>, diesel::types::Text), diesel::pg::Pg> for models::Tst1
= note: required because of the requirements on the impl of diesel::LoadQuery<diesel::PgConnection, models::Tst1> for diesel::query_builder::insert_statement::InsertStatement<schema::__diesel_infer_schema::infer_tst1::tst1::table, (diesel::insertable::ColumnInsertValue<schema::__diesel_infer_schema::infer_tst1::tst1::columns::methd, diesel::expression::bound::Bound<diesel::types::Text, &std::string::String>>,)>
error: aborting due to previous error

@greenpdx Your issue is slightly different than that one described here. You are trying to load a nullable sql type (Nullable<Timestamp>) into a not nullable rust type (NaiveDateTime). Diesel does not support such mappings to prevent errors. Instead try to change the rust type into Option<NaiveDateTime>.

I encountered another similiar error: *const str: diesel::deserialize::FromSql<diesel::sql_types::Timestamp, diesel::pg::Pg, anyone knows how to fix it?

@GopherJ That error message indicates that you trying to load a Timestamp into a string which is not supported. You need to enable the chrono feature and use chrono::NaiveDateTime for this.

I get this issue still, using chrono etc.
use super::schema::trades;
use chrono;
#[derive(Queryable,Debug)]
pub struct Trade {
pub id: i32,
pub cusip: Option,
pub sedol: Option,
pub isin: Option,
pub ticker: Option,
pub trade_date: chrono::NaiveDateTime,
pub published: bool,
}
#[derive(Insertable,Debug)]
#[table_name = "trades"]
pub struct NewTrade<'a> {
pub cusip: &'a str,
pub sedol: &'a str,
pub isin: &'a str,
pub ticker: &'a str,
pub trade_date: chrono::NaiveDateTime,
}

error[E0277]: the trait bound chrono::NaiveDateTime: diesel::Expression is not satisfied
--> src/models.rs:32:10
|
32 | #[derive(Insertable,Debug)]
|          ^^^^^^^^^^ the trait diesel::Expression is not implemented for chrono::NaiveDateTime
|
= note: required because of the requirements on the impl of diesel::expression::AsExpression<diesel::sql_types::Timestamp> for chrono::NaiveDateTime
error[E0277]: the trait bound chrono::NaiveDateTime: diesel::Expression is not satisfied
--> src/models.rs:32:10
|
32 | #[derive(Insertable,Debug)]
|          ^^^^^^^^^^ the trait diesel::Expression is not implemented for chrono::NaiveDateTime
|
= note: required because of the requirements on the impl of diesel::Expression for &chrono::NaiveDateTime
= note: required because of the requirements on the impl of diesel::expression::AsExpression<diesel::sql_types::Timestamp> for &chrono::NaiveDateTime
error[E0277]: the trait bound chrono::NaiveDateTime: diesel::Expression is not satisfied
--> src/models.rs:32:10
|
32 | #[derive(Insertable,Debug)]
|          ^^^^^^^^^^ the trait diesel::Expression is not implemented for chrono::NaiveDateTime
|
= note: required because of the requirements on the impl of diesel::Expression for &'insert chrono::NaiveDateTime
= note: required because of the requirements on the impl of diesel::expression::AsExpression<diesel::sql_types::Timestamp> for &'insert chrono::NaiveDateTime
error: aborting due to 3 previous errors

@dosterthebernese Our issue tracker is used to track bugs and feature request. For asking questions please use our gitter channel


startup error with latest MassTransit and Quartz

I'm migrating to MT3 along with newest MongoDB/Quartz/etc., and am having issues getting your mongodb connector working.  I was using https://github.com/Nanoko/quartznet-mongodb previously, but want Mongodb 2.x driver support.
When it tries to resolve the scheduler from the factor I get:
The memberInfo argument must be for class JobKey, but was for class Key`1.\r\nParameter name:
memberInfo

It seems to be an issue coming from the BsonSerializer, which doesn't make a ton of sense to me as I'm pointing this thing at a fresh DB (was thinking it might be from existing records in the old db at first).
Thoughts?
I'm using the following setup:
var properties = new NameValueCollection
            {
                [StdSchedulerFactory.PropertySchedulerInstanceName] = "MassTransit_QuartzNet",
                [StdSchedulerFactory.PropertySchedulerInstanceId] = $"{Environment.MachineName}-{Guid.NewGuid()}",
                [StdSchedulerFactory.PropertyJobStoreType] = typeof(MongoDbJobStore).AssemblyQualifiedName,
                [
                    $"{StdSchedulerFactory.PropertyJobStorePrefix}.{StdSchedulerFactory.PropertyDataSourceConnectionString}"
                    ] = ConfigurationManager.ConnectionStrings["quartznet-mongodb"].ConnectionString,
                [$"{StdSchedulerFactory.PropertyJobStorePrefix}.collectionPrefix"] = "prefix"
            };
            // I treat the database in the connection string as the one you want to connect to
            // The prefix is optional

            var factory = new StdSchedulerFactory(properties);

with app.config records as follows:
 <connectionStrings>
    <add name="quartznet-mongodb" connectionString="mongodb://quartznet:passwordhere@mongo1/quartznet" />
  </connectionStrings>
  <quartz>
    <add key="quartz.scheduler.instanceName" value="MassTransit_QuartzNet" />
    <add key="quartz.threadPool.type" value="Quartz.Simpl.SimpleThreadPool, Quartz" />
    <add key="quartz.threadPool.threadCount" value="4" />
    <add key="quartz.threadPool.threadPriority" value="2" />
    <add key="quartz.jobStore.misfireThreshold" value="60000" />
    <add key="quartz.jobStore.type" value="Quartz.Spi.MongoDbJobStore.MongoDbJobStore, Quartz.Spi.MongoDbJobStore" />
  </quartz>

Do you have a stack trace at all? Are you sure there is no existing data in the database?

There is an existing database called "quartz" which does have old data (with "MassTransit" prefix, which is why i updated this to be "MassTransit_QuartzNet"), and that was my original thought, however I updated the connection string to point to "quartznet" instead.  Is there maybe something hardcoded to "quartz" database name?  Let me dig up a stacktrack for you, and thanks so much for the quick response.

Here is the inner exception stacktrace
   at MongoDB.Bson.Serialization.BsonClassMap.EnsureMemberInfoIsForThisClass(MemberInfo memberInfo)
   at MongoDB.Bson.Serialization.BsonClassMap.MapMember(MemberInfo memberInfo)
   at MongoDB.Bson.Serialization.Conventions.ImmutableTypeClassMapConvention.Apply(BsonClassMap classMap)
   at MongoDB.Bson.Serialization.Conventions.ConventionRunner.Apply(BsonClassMap classMap)
   at MongoDB.Bson.Serialization.BsonClassMap.AutoMapClass()
   at MongoDB.Bson.Serialization.BsonClassMap.AutoMap()
   at Quartz.Spi.MongoDbJobStore.Repositories.JobStoreClassMap.<>c.<RegisterClassMaps>b__0_2(BsonClassMap`1 map) in D:\github\georiot\core\sln\backend\mongodb-quartz-net\Quartz.Spi.MongoDbJobStore\Repositories\JobStoreClassMap.cs:line 35
   at MongoDB.Bson.Serialization.BsonClassMap`1..ctor(Action`1 classMapInitializer)
   at MongoDB.Bson.Serialization.BsonClassMap.RegisterClassMap[TClass](Action`1 classMapInitializer)
   at Quartz.Spi.MongoDbJobStore.Repositories.JobStoreClassMap.RegisterClassMaps() in D:\github\georiot\core\sln\backend\mongodb-quartz-net\Quartz.Spi.MongoDbJobStore\Repositories\JobStoreClassMap.cs:line 33
   at Quartz.Spi.MongoDbJobStore.MongoDbJobStore..cctor() in D:\github\georiot\core\sln\backend\mongodb-quartz-net\Quartz.Spi.MongoDbJobStore\MongoDbJobStore.cs:line 45

@chrisdrobison it must be breaking before even getting to mongo, as if i kill my mongo connection string (replaced it with bad hostname) I get the same error.  Could it have something to do with the actual message its trying to process?
here is a sample message payload (queued through MT3 SchedulePublish method:
{

  "messageId": "4be20000-5db3-0015-77d1-08d435b8f840",

  "conversationId": "4be20000-5db3-0015-fd7b-08d435b8f841",

  "sourceAddress": "rabbitmq://mq1-production/bus-PDESK-Test.QueueDelayedMessage.vshost-jxtyyyn7scybko8wbdkdmq8z84?durable=false&autodelete=true",

  "destinationAddress": "rabbitmq://mq1-production/ha.masstransit_quartz",

  "messageType": [

    "urn:message:MassTransit.Scheduling:ScheduleMessage[[Test.QueueDelayedMessage:TestMessage]]",

    "urn:message:MassTransit.Scheduling:ScheduleMessage"

  ],

  "message": {

    "correlationId": "4be20000-5db3-0015-1a35-08d435b8f81f",

    "scheduledTime": "2017-01-05T22:19:56.9030225Z",

    "payloadType": [

      "urn:message:Test.QueueDelayedMessage:TestMessage"

    ],

    "destination": "rabbitmq://mq1-production/Test.QueueDelayedMessage:TestMessage",

    "payload": {

      "correlationGuid": "b3dbd2d4-4a4a-4cba-a145-a1b63deaa2a5",

      "createdAt": "2017-01-05T22:19:51.9025213Z",

      "text": "testing delayed message publishing!"

    }

  },

  "headers": {},

  "host": {

    "machineName": "PDESK",

    "processName": "Test.QueueDelayedMessage.vshost",

    "processId": 17580,

    "assembly": "Test.QueueDelayedMessage",

    "assemblyVersion": "<IP_ADDRESS>",

    "frameworkVersion": "4.0.30319.42000",

    "massTransitVersion": "<IP_ADDRESS>9",

    "operatingSystemVersion": "Microsoft Windows NT 6.2.9200.0"

  }

}

OOOH.  I pulled in your code and built it from github, and used that instead of Nuget package, and here is where it breaks..
in JobStoreClassMap.cs:
BsonClassMap.RegisterClassMap<JobKey>(map =>
            {
                map.AutoMap();
                map.MapCreator(jobKey => new JobKey(jobKey.Name));
                map.MapCreator(jobKey => new JobKey(jobKey.Name, jobKey.Group));
            });

Interesting. All that is working for me.

No, this breaks during initialization of the scheduler, before i get to do anything on my own even:
            {
                [StdSchedulerFactory.PropertySchedulerInstanceName] = "MassTransit_QuartzNet",
                [StdSchedulerFactory.PropertySchedulerInstanceId] = $"{Environment.MachineName}-{Guid.NewGuid()}",
                [StdSchedulerFactory.PropertyJobStoreType] = typeof(MongoDbJobStore).AssemblyQualifiedName,
                [
                    $"{StdSchedulerFactory.PropertyJobStorePrefix}.{StdSchedulerFactory.PropertyDataSourceConnectionString}"
                    ] = ConfigurationManager.ConnectionStrings["quartznet-mongodb"].ConnectionString,
                [$"{StdSchedulerFactory.PropertyJobStorePrefix}.collectionPrefix"] = "prefix"
            };
            // I treat the database in the connection string as the one you want to connect to
            // The prefix is optional

            var factory = new StdSchedulerFactory(properties);

            var MongoUrl = new MongoUrl(ConfigurationManager.ConnectionStrings["quartznet-mongodb"].ConnectionString);

            //BREAKS HERE
            var scheduler2 = factory.GetScheduler();
``

Which version of the driver are you using?

I have latest nuget for everything, and git master from yesterday on your driver.

Try an older 2.x driver and see if it works.
On Fri, Jan 6, 2017 at 1:06 PM Jesse Pasichnyk<EMAIL_ADDRESS>wrote:

I have latest nuget for everything, and git master from yesterday on your
driver.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/chrisdrobison/mongodb-quartz-net/issues/4#issuecomment-270993306,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABu24fgeG7fmFFV6KXJh25CGjrEQDeKsks5rPp7KgaJpZM4LcR_T
.

@chrisdrobison Wow, good call.  Running with MongoDB.* 2.2.0 NuGet packages, it runs! :)

I'll have to get some input from the driver authors about this issue then.
On Fri, Jan 6, 2017 at 4:05 PM Jesse Pasichnyk<EMAIL_ADDRESS>wrote:

@chrisdrobison https://github.com/chrisdrobison Wow, good call. Running
with MongoDB.* 2.2.0 NuGet packages, it runs! :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/chrisdrobison/mongodb-quartz-net/issues/4#issuecomment-271034577,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABu24VV9uu2u3R4flmqTn8LpHa0JvDzUks5rPsjRgaJpZM4LcR_T
.


pull request for e2e test at 01/14/2017 12:56:12

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


add nextPos in canal.RowsEvent

ref #247
hi, I add the nextPos in canal.RowsEvent, would you please check if it is ok?

@ideamean I will close this pr. If you still need it, welcome to open and update it again


q2-Songbird: plotting regression-biplots issue

Hi Jamie,
Unless I am misunderstanding something, I think there might be an issue with the QIIME2 Songbird plugin regarding making the regression biplot.  When I use your RedSea tutorial dataset, the qiime songbird multinomial function appears to run correctly, but when I go to the plug the regression-biplot.qza into anime emperor biplot I get an error:
(qiime2-2019.1) jobakerlt-osx:march-q2songbird jobaker$ qiime songbird multinomial \

--i-table redsea.qza
--m-metadata-file redsea_metadata.txt
--p-formula "Depth+Temperature+Salinity+Oxygen+Fluorescence+Nitrate"
--o-differentials redsea_differentials.qza
--o-regression-stats redsea_regression-stats.qza
--o-regression-biplot redsea_regression-biplot.qza
--verbose
2019-05-07 20:40:05.079391: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2019-05-07 20:40:05.079660: I tensorflow/core/common_runtime/process_util.cc:69] Creating new thread pool with default inter op setting: 8. Tune using inter_op_parallelism_threads for best performance.
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8000/8000 [00:10<00:00, 731.36it/s]
Saved FeatureData[Differential] to: redsea_differentials.qza
Saved SampleData[SongbirdStats] to: redsea_regression-stats.qza
Saved PCoAResults % Properties(['biplot']) to: redsea_regression-biplot.qza
(qiime2-2019.1) jobakerlt-osx:march-q2songbird jobaker$ qiime emperor biplot
--i-biplot redsea_regression-biplot.qza
--m-sample-metadata-file redsea_metadata.txt
--p-number-of-features 7
--o-visualization redsea_emperor-biplot
Plugin error from emperor:

None of the sample identifiers match between the metadata and the coordinates. Verify that you are using metadata and coordinates corresponding to the same dataset.
Debug info has been saved to /var/folders/rr/2hbbkrqs5mn86rnt2s4yywsr0003kj/T/qiime2-q2cli-err-y6jdxgcr.log
I noticed this error initially when running my own dataset, using my same OTU table and metadata table that had worked with making a biplot from DEICODE (i.e. the biplot didn't give a similar error then).  Since your RedSea tutorial data had the same issue I don't actually think it's a lack of match up between the OTU table and metadata.  If it helps, my qiime info is here:
System versions
Python version: 3.6.5
QIIME 2 release: 2019.1
QIIME 2 version: 2019.1.0
q2cli version: 2019.1.0
Installed plugins
alignment: 2019.1.0
composition: 2019.1.0
cutadapt: 2019.1.0
dada2: 2019.1.0
deblur: 2019.1.0
deicode: 0.1.6
demux: 2019.1.0
diversity: 2019.1.0
emperor: 2019.1.0
feature-classifier: 2019.1.0
feature-table: 2019.1.0
fragment-insertion: 2019.1.0
gneiss: 2019.1.0
longitudinal: 2019.1.0
metadata: 2019.1.0
phylogeny: 2019.1.0
quality-control: 2019.1.0
quality-filter: 2019.1.0
rankratioviz: 0.0.0
sample-classifier: 2019.1.0
songbird: 0.8.2
taxa: 2019.1.0
types: 2019.1.0
vsearch: 2019.1.0
Thanks!

I bet that it is stemming from the confusion due to sample metadata
The actual command should be this
qiime emperor biplot \
> --i-biplot regression-biplot.qza \
> --m-sample-metadata-file data/redsea/feature_metadata.txt \
> --p-ignore-missing-samples \
> --p-number-of-features 7 \
> --o-visualization emperor-biplot

What's actually happening here is that the sample-metadata doesn't refer to the actual metadata for the samples, it refers to the feature metadata (i.e. taxonomies, genome information, ...)
I agree - the parameters are a little confusing.  Maybe a quick fix is to introduce additional biplot commands in Emperor to help differentiate between the different types of metadata?  I'm open for suggestions. CC @ElDeveloper

I agree, perhaps the parameter names should be more specific to the "arrows" and "points" and instead be --m-point-metadata-file and --m-arrow-metadata-file. With all the different types of data that can be plotted as a biplot making these parameters "sample metadata" and "feature metadata" can be a bit confusing for some use-cases.

Thanks! So, I got it to work with your tutorial by using the feature metadata file instead of the sample metadata file, but with my data, I keep getting the same error.
In my dataset, the features are species names.  I don't really have any metadata for my species, so I just made a tsv with the species names in the first column and kingdom (bacteria or virus) in the 2nd column (just to have something).  When giving the emperor biplot this table, I get the same error.  As far as I can tell, they match up exactly, but I still get the error...any ideas?
I've attached my two metadata files and otu table.  Thanks so much for your help.
otu_table.txt
feature_metadata.txt
sample_metadata.txt

Ok. Can you post your biplot as well so that I can reproduce the error?
On Wed, May 8, 2019, 5:24 PM Jonathon L. Baker<EMAIL_ADDRESS>wrote:

Thanks! So, I got it to work with your tutorial by using the feature
metadata file instead of the sample metadata file, but with my data, I keep
getting the same error.
In my dataset, the features are species names. I don't really have any
metadata for my species, so I just made a tsv with the species names in the
first column and kingdom (bacteria or virus) in the 2nd column (just to
have something). When giving the emperor biplot this table, I get the same
error. As far as I can tell, they match up exactly, but I still get the
error...any ideas?
I've attached my two metadata files and otu table. Thanks so much for your
help.
otu_table.txt
https://github.com/biocore/songbird/files/3159258/otu_table.txt
feature_metadata.txt
https://github.com/biocore/songbird/files/3159259/feature_metadata.txt
sample_metadata.txt
https://github.com/biocore/songbird/files/3159260/sample_metadata.txt
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/biocore/songbird/issues/54#issuecomment-490656395,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA75VXJKTUGOGPVRKM4K4PLPUNAKNANCNFSM4HLTLSZQ
.

You mean the output from giime songbird multinomial? Let me know if you mean something else
regression-biplot.qza.zip


Partial Updates on Custom Fields

Steps to recreate

Login to Lazymine
Select several tasks from Searchbox
Add Comment, Activity and Hours for all of the tasks
Add required Custom Field values to some of the tasks
Click 'Update'
Verify data on Redmine server

Expected Result

Should not post any data to Redmine server.

Actual Result

Posts data to the Redmine server for the tasks which has the required Custom Field data.

Fixed


Fix null pointer exception caused by clearing member variable

R: @prodriguezdefino

I notice that nullness checking is enabled for these files, right? Which was the variable that was incorrectly annotated, and where was it dereferenced? Before cherrypicking this to the release, I'd like to enable nullness where it needs to be to prevent further regression.

this wasn't actually a null pointer exception (the title of the PR is misleading). Rather it was a failure in Preconditions.checkStateNotNull


Excuse me, what is this question

[2024-07-02 18:19:39,263] Frame.goto() missing 1 required positional argument: 'url'
Traceback (most recent call last):
File "F:\vfs-appointment-bot-main\vfs-appointment-bot-main\vfs_appointment_bot\main.py", line 89, in main
appointment_found = vfs_bot.run(args)
File "F:\vfs-appointment-bot-main\vfs-appointment-bot-main\vfs_appointment_bot\vfs_bot\vfs_bot.py", line 80, in run
page.goto(vfs_url)
File "F:\vfs-appointment-bot-main\vfs-appointment-bot-main\env\lib\site-packages\playwright\sync_api_generated.py", line 8686, in goto
self._sync(
File "F:\vfs-appointment-bot-main\vfs-appointment-bot-main\env\lib\site-packages\playwright_impl_sync_base.py", line 115, in _sync
return task.result()
File "F:\vfs-appointment-bot-main\vfs-appointment-bot-main\env\lib\site-packages\playwright_impl_page.py", line 519, in goto
return await self._main_frame.goto(**locals_to_params(locals()))
TypeError: Frame.goto() missing 1 required positional argument: 'url'

I'm facing the same issue


Issue with uWSGI example after code refactoring

After upgrading PowerDNS-Admin today from a codebase before the refactoring I can't start the application via uwsgi anymore. The Wiki example has mount = /pdns=run.py which I think should now be mount = /pdns=powerdnsadmin:app but my service refuses to start and gives the following error:
unable to load app 0 (mountpoint='/pdns') (callable not found or import error)

Does anyone have a pointer what the mount directive should look like with the new code structure? Or am I maybe searching for the wrong thing and this is connected with the additional FLASK_APP and FLASK_CONFIG environment variables that uwsgi needs to access but can't?
Thanks in advance!

If I leave mount = /pdns=run.py and change callable to create_app the application starts but the logfile gives me another error:
TypeError: create_app() takes from 0 to 1 positional arguments but 2 were given

Put the following glue code in a run_uwsgi.py:
#!/usr/bin/env python3

from powerdnsadmin import create_app

# ----------------------------------------------------------------------
def main(*args, **kwargs):
    app = create_app()
    return app(*args, **kwargs)

And then use mount = /pdns=run_uwsgi:main.

Thank you very much @eht16 that worked perfectly :+1:
Do you want to PR the changes or should I?

Feel free to do. Fine for me.

Please forget the script I posted, it doesn't work properly as on each request, the app is completely started again.
The proper way to start PDNSA using uwsgi is: wsgi ... --mount /='powerdnsadmin:create_app()'
No additional script is necessary, create_app() is already there.

Ah, that's one option I didn't try! :man_facepalming:
Thanks for re-checking. I will amend the Wiki here so people are aware of the change.


Display branding for Alibaba infrastructure

Signed-off-by: Sho Weimer<EMAIL_ADDRESS>https://github.com/stolostron/multicloud-operators-foundation/pull/457/files

/lgtm


out of bounds

cppcheck reports:
When i==arg.size(), arg[i] is out of bounds. on line 554 in clara.h

I'm going to close this as the code has been entirely rewritten (although, if memory serves, that issue was fixed in 0.x anyway)


Configuration: show number of violated rules when using auto-configuration

Fixes #1125

Great work. I need to pull it down and verify soon. A test would be great. Any chance you could whip one up?

Not sure what exactly should be tested -- probably not that "#Rules" shows up in the output, but maybe that the reported number of violations is correct? (which boils down to a verification of getUniqueErrorNames(errors))

Coverage remained the same at 98.95% when pulling 16d8cf60d7b5b65a179cdc3a6c04c5728ea7aaad on fubu:auto-config-rule-count into 84ab5a3f2bcdc46c80eba5b17c00d5e01295be31 on jscs-dev:master.

Agreed about the testing being not too obvious. The best way to test this is to fix the fixture file; currently it's the source code for the config generator, which I thought was cute, but isn't.
We need a control file that has a guaranteed results set of errors and violated rules. Then we know for sure what the console output should look like and can split lines to assert the number of violated rules.
For now, I'd say a manual verification is okay since coverage remained the same.

Coverage increased (+0.0%) to 98.96% when pulling 8becf0228923d05c2f6e18eb37eb78aab2e6e031 on fubu:auto-config-rule-count into 84ab5a3f2bcdc46c80eba5b17c00d5e01295be31 on jscs-dev:master.

LGTM

@fubu Great work here. Mind rebasing off of master and then we'll land this.

I don't, but my git-skills are a bit rusty. What exactely should I do? :)

Coverage increased (+0.0%) to 98.96% when pulling e471650a73f89568bc1fa725df91f246746170c9 on fubu:auto-config-rule-count into 89268e5a6e8cedd5157014a93fdaa8d11ecd5ffb on jscs-dev:master.

git pull - -  rebase upstream master. Assuming you did: git remote add upstream the_jscs_repo_ssh_url.
Let me know if that doesn't work.

Yep, managed to do it in the meantime, thanks! Don't know about the failed build though, the AppVeyor page takes forever to load...

@fubu I can't get that build to load on appveyor. Any chance you could force push up an insignificant change to trigger a rebuild?

You can see a part of errors here: https://ci.appveyor.com/project/mdevils/node-jscs/build/154
https://ci.appveyor.com/api/buildjobs/1v3k79cy024mr1ym/log — and full log here (3.4Mb)

K.  Thanks @zxqfox

Great work @fubu. The extra info that you added is really helpful. Thanks for contributing.


V8: file manager on mobile

complete useless:

additionally this new "right mouse click" thing breaks the usability on all touch devices.
at least on my iPhone and iPad the File Manager is not useable.
for me personally a right-click functionality on a web-app is bad practice.

There is a bulk menu that should be a better solution on touch devices - just click as many files as you want and click the drop down button. Unfortunately I think the broken layout on mobile is obscuring it.

Sounds like the menu in the file manager could use mobile touch event bindings as well

@KorvinSzanto I've tweaked some CSS and the file manager, page search and user search look a lot better (although still not perfect.) However there are some issues

[ ] Can we get rid of dragging/dropping images on mobile? It intercepts the scrolling action (at least in the simulator.) If we need to enable this functionality on mobile we'll probably just do it through dialogs.
[ ] Mobile dialogs appear to bust outside the dimensions of my simulator. This wasn't always the case. I'm wondering if something to do with upgrading jQuery UI broke something
[ ] Anything else you see?

Will do


modify some errors of plurality

Signed-off-by: liwenqi<EMAIL_ADDRESS>
- What I did
- How I did it
- How to verify it
- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

Please consider squashing https://github.com/docker/docker/pull/29620

Windows CI is still pending, but this doesn't effect anything CI related, and lint/vet validations have all passed.
Merging.


Improve some gamm error messages that came up in QA

What is the purpose of the change
Improve some GAMM error messages
Documentation and Release Note

Does this pull request introduce a new feature or user-facing behavior changes? yes
Is a relevant changelog entry added to the Unreleased section in CHANGELOG.md? yes
How is the feature or change documented? N/A

Does changing the error message for v12 potentially make this state breaking? Unsure what gets merkelized here

nah the string herre isn't merkelized, only the error code


Xrm.WebApi TypeScript Samples for Model Driven Apps

This PR includes samples on how to use the Xrm.WebApi from inside a model-driven app using TypeScript.
See the readme on how to build and run the samples

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: scottdurow sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

@scottdurow Thanks for the sample. We will review it and let you know

Sent this to the feature team for review.
cc: @sidhartg

Any feedback on this PR?

@scottdurow
Finally closing this as we discussed.


cofight Server missing

Hello,
is there a basic server code ?
I set up a http-server that replies "1" to the eight digit phone-number.
But what about the activation code ?
grateful for any help
mateng

Hello and thanks for the interest In our project!
We have a repository for the server-side: https://github.com/CoFight-19/cofight-cfml
The code has not yet been uploaded. You can subscribe to the repo to get notified.


Feat: I will add an about us page .

Describe the feature
I will add a About us page so that user can get the abstract view of the website .
Add ScreenShots
This helps the user to know or get details about the website .
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

@Ayushmaanagarwal1211 as of now, we think we don't need this, will let you know in future, if there is need


Ignore more celerybeat-schedule files

Since celerybeat uses anydbm (via shelve), its data file might have one of a number of file extensions. It's actually pretty hard to figure out what those file extensions might be, but it's easy enough to keep them out of git with a wildcard.
P.S. I don't know what the celery_beat.py.db file is about, so I'm not touching that.
Reticketed from https://github.com/zkraime/osf.io/commit/a6d27826a247615b5de4838d76bf27b714aa144f#commitcomment-16682405.

Thanks


Skip on comment

Description:

Related issue(s):
Fixes #
Notes for reviewer:

Checklist

[ ] Documented (Code comments, README, etc.)
[ ] Tested (unit, integration, etc.)

test comment


Trim function not parsing correctly

Describe the bug
When I use the trim function it struggles to remove values such as '"' and ',' - I can't get these values out. When testing this on an online regex  - like regexr - it removes the quotations
To get around this I am using the mv-expand method
To Reproduce
Try the following trim:
SecurityIncident | extend Tactics = trim(@'[["\]]+',tostring(AdditionalData.tactics))
Expected behavior
No unexpected quotes
Screenshots
Expected:

Actual result:

Additional context
To get around this I'm using the following method of parsing:
SecurityIncident | mv-expand AdditionalData | extend ProductName = parse_json(tostring(AdditionalData.alertProductNames)) | mv-expand ProductName | extend tactics = parse_json(tostring(AdditionalData.tactics)) | mv-expand tactics
I'd like to get unique values using the trim method.

Hey @Lodewyk-Git, not really an answer as to why trim isn't removing the double quotes. But I believe I've found a possible workaround using the strcat_array function
SecurityIncident | extend Tactics = strcat_array(AdditionalData.tactics,",")
The result is a Tactics columns with no ",[ or ] in it.

Thank you @tvuylsteke! This is a very helpful function.

@Lodewyk-Git - I don't think Trim will do what you are asking and isn't designed for it based on documentation.  Trim operates only on the start and end of the string, which is what it appears to be doing in your testing - https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/trimfunction.
I think that replace_regex() does what you want - https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/replace-regex-function
SecurityIncident
| project Title, AdditionalData
| where AdditionalData.tactics != '[]'
| where array_length( AdditionalData.tactics) > 1 //just here to test with more than one tactic, remove
| extend tactics = AdditionalData.tactics
| extend lngth = array_length(AdditionalData.tactics)
| project Title, tactics
| extend Tactics_replace = replace_regex(tostring(tactics), @'[["\]]+', '')


Improve explainStr message

Hello @prayerslayer! First of all thanks for the port!
I am playing with it from TypeScript and have the following spec:
/**
 * Return true if there are no spaces
 */
export function hasNoSpaces(s: string): boolean {
  const regex = new RegExp(/\s+/);
  return !regex.test(s);
}

export const AggregateSegmentSpec = S.spec.and("s_aggregateSegment", S.spec.string, hasNoSpaces);

When it fails the message is:
"s_aggregateSegment → hasNoSpaces: hasNoSpaces failed for undefined at []."
Which is a bit cryptic and probably has a bug in it (see undefined).
I'll try to have a look but can't promise anything :smile:

Haha, yeah. One of this library's goals was to unify error messages, but I couldn't focus on that part yet (as we first need to find the errors). Would definitely appreciate improvements and suggestions!

So what I see if I dump problems in explainData is:
[ { path: [],
    via: [ 'DataPathSpec', 'p' ],
    value: ... ]

and the getIn(value, problem.path) of course always returns undefined.
Maybe spec.and does not build the correct path for the specs.
Debugging time is over for now, just wanted to write down this little discovery.

So the working idea of path was that it includes the path in a nested structure to access the failing value. If there is no nested structure involved, it's always empty. I'm not sure what to do, should it in your example contain the index of the offending character?

@prayerslayer I see now. My predicate is arbitrary so what I think clojure.spec is doing is just to show the value that fails. It we had a way for a predicate to specify a failing "path" we would be actually better than clojure.spec at this :smile:

Alright, so I think all there is to do is to exclude path from the explanation when it's empty. I'd like it to point to a character in a string, but think it's not easily doable with native JS regexes. We would need a regex engine like regex101.com has, where we can access intermediate steps.

Yeah removing that would solve thing and make it less confusing I guess. Good solution for now!

Is this now fixed?

I think so, the message got improved. The regex specific message can be handled in another issue I guess. Thanks a lot for the fix!


Sas

Added Account SAS capability and Service SAS capability for Blob and Container clients.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: gapra-msft sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Can one of the admins verify this patch?

Has this been reviewed by @rickle-msft or @jaschrep-msft ?

@JonathanGiles. James and I both reviewed it before the PR was created. The SAS generation logic and testing are up to snuff, and we thought it was a reasonable api surface for preview, but we are open to suggestions.

No, I just wanted to be clear on the status. I'll take a good look in the next days. Thanks

@JonathanGiles I don't think so. That's actually what we had in v11, and I don't think the result was very good. Customers had to set a bunch of fields on ServiceSASSignature values, and use that to eventually generate an encoded query string which they had to attach onto a URL. We had some complaints that customers didn't even think we had support for SAS because it was so difficult to find. Perhaps some clearer naming, etc. may have helped, and I think there's room for improvement, but I think there has to be some sort of connection to it on the Client for discoverability.


Pjax not working when clicking on a link within DataTables

There seems to be an issue with Pjax not detecting links within DataTables. Any ideas how to get around this?

Please link to a CodePen demonstrating the issue.

https://codepen.io/xenace/project/editor/ZgPgEK#0

What's not working? Everything looks fine to me, and the links work on all pages.

Sorry was just testing, narrowed it down to it being an issue on paginated links. If you switch to a different page and select a link it does not load through Pjax, just as a regular page load. I'm guessing that it's something to do with those links not being in the DOM when loaded.

I'm guessing that it's something to do with those links not being in the DOM when loaded.

It sounds like the elements are being dynamically generated. Try calling pjax.parseDOM(document) after switching pages.

That seems to have partly solved the issue however when clicking on the pagination links Pjax is trying to load the root page because the buttons have a href value of # i got around this before by using javascript:void(0) in the href value but would then need to override the datatable functionality to do get around this. Is there any way to tell Pjax to ignore href with # value?

Ignore last message, resolved it by setting elements to a:not(.page-link)

Just come here to add that there is actually a pjax.refresh() method that is designed for situations like this where you need to make Pjax aware of new DOM elements which we should document properly. It is just a wrapper around pjax.parseDOM(), so @BehindTheMath's advice is still correct.
https://github.com/MoOx/pjax/blob/216686696712d4a3cfe1ebe9b43c3fb5c8e2dd30/index.js#L69-L71


Filter branches by shape - GUI update

Hi @jni,
Filtering branches by shape is useful for analysing the skeleton - this has been implemented within the GUI functionality within this branch so that users do not need to write Python code.

Coverage remained the same at 99.396% when pulling e3f7ef2c94e08b4836aab1dc78690b8fa43b6980 on ajblanch:shapefilterPR into 93be0b3fc2c4e080dbabd6a0fb6e5832423af2e8 on jni:master.

@ajblanch great! I'd forgotten that this actually writes to a separate sheet in the excel file. That makes it pretty nice and non-destructive. =)


Pardot destination

Description of the change

Description here

Type of change

[ ] Bug fix (non-breaking change that fixes an issue)
[ ] New feature (non-breaking change that adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Related issues

Fix #1

Checklists
Development

[ ] Lint rules pass locally
[ ] The code changed/added as part of this pull request has been covered with tests
[ ] All tests related to the changed code pass in development

Code review

[ ]  This pull request has a descriptive title and information useful to a reviewer. There may be a screenshot or screencast attached
[ ] "Ready for review" label attached to the PR and reviewers mentioned in a comment
[ ] Changes have been reviewed by at least one other engineer
[ ] Issue from task tracker has a link to this pull request

I see that we aren't mentioning Pardot is currently only available for **Production** environment Because, we currently are providing support only for that

@abandonned Can we also verify this with our research spec & see if we are mentioning all the specifics that'd be needed for the user ?


Improvements to list method on aws and gcp

Codecov Report
Attention: Patch coverage is 20.00000% with 8 lines in your changes are missing coverage. Please review.

Project coverage is 85.98%. Comparing base (a8d6d33) to head (9f52b0a).

Files
Patch %
Lines

runner_manager/backend/aws.py
20.00%
4 Missing :warning:

runner_manager/backend/gcloud.py
20.00%
4 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #583      +/-   ##
==========================================
- Coverage   86.11%   85.98%   -0.13%
==========================================
  Files          31       31
  Lines        1246     1249       +3
==========================================
+ Hits         1073     1074       +1
- Misses        173      175       +2

Flag
Coverage Δ

api
66.61% <20.00%> (-0.09%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Adding TURN API to RTC Engine

Related pull requests:

https://github.com/membraneframework/stun/pull/1
https://github.com/membraneframework/ex_libnice/pull/19
https://github.com/membraneframework/membrane_demo/pull/150
https://github.com/membraneframework/membrane_ice_plugin/pull/31


Handle when counter metric has _total suffix

Some of the Prometheus client libraries automatically append _total to the names of any counters. This makes the recording rules work, whether the metric is called function_calls_count or function_calls_count_total.

Tested this on my local python set up and finally did get the High Error Rate SLO


Temporarily install https certificate

Can you temporarily install the certificate for the platform using letsencrypt.org or, by default, certbot.eff.org to keep the links of the users active? I see that your domain (sourcerer.io) expires 2021-06-18

i also got the ssl error today

Just saw #632


Update README.md

:tada: This PR is included in version 1.111.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket: