Implement an abstraction for wamp transports

The session API is templated and requires that both an input stream and output stream be provided. For example:
autobahn::session<boost::asio::local::stream_protocol::socket, boost::asio::local::stream_protocol::socket>
autobahn::session<boost::asio::ip::tcp::socket, boost::asio::ip::tcp::socket>
autobahn::session<int, int>

Calls to boost::asio::write, boost::asio::read, and boost::asio::async_read are then issued internally as required to send and receive messages. This is restricting such that the templated types must be compatible with boost::asio semantics for reading and writing data. In the near future, support will likely be added for Websocket, HTTP/2, and a native application transport to call directly into a WAMP router. In order to do this, we will need to be able to have the session API internally write to these types of transports. As a result we need to provide further abstraction in place of the templated streams that are provided to the session. For example:
class wamp_stream
{
public:
    wamp_stream();
    virtual wamp_stream();

    virtual int read(char* buffer, int length) = 0;
    virtual void async_read(char* buffer, int length, read_handler handler) = 0;
    virtual void write(const char* buffer, int length) = 0;
};

class wamp_websocket_stream : public wamp_stream
{
...
};

class wamp_http2_stream : public wamp_stream
{
...
};

class wamp_native_stream : public wamp_stream
{
...
};

Then we can change the session API to no longer require templates for the input and output streams and instead pass in objects of the desired types:
autobahn::session(boost::asio::io_service& io, wamp_stream& in, wamp_stream& out, bool debug=false)

As an added benefit, this would also allow the session API to participate in a factory pattern which could be desireable for implementations requiring dynamic session generation based on transport type.

I am going to start work on this as I now need this to avoid a bunch of template issues.

This is restricting such that the templated types must be compatible with boost::asio semantics for reading and writing data.

So the semantics of above wamp_stream abstraction would be different from a asio stream? In what way? Because wamp_stream basically is a byte stream abstraction, right?

@oberstet by abstracting an interface we can actually hide implementation details for things like native c++-to-c++ router calls, websocket communication, http2 communication, etc. Also, having template parameters on the session class is a bit limiting. For example, if you look at how I did components, I had to templatize the component class. As a result, I can't have something generically accept a component. For example:
class system_client
{
public:
    my_client(autobahn::wamp_tcp_component& component);

    status get_network_interfaces(network_interfaces& interfaces);
    .
    .
    .
};

If I want to reuse this client API I would need to templatize it to accept the component type and even then it limits me to static types. Alternatively, if we could get rid of the template requirement I could write this simply as:
class system_client
{
public:
    my_client(autobahn::wamp_component& component);

    status get_network_interfaces(network_interfaces& interfaces);
    .
    .
    .
};

Taking this approach I can then pass whatever component type I want to the client without the need for templates. This allows me to reuse the client in a number of scenarios where I may be using TCP, UDS, Websocket, Http2, etc as the transport type and I can determine this dynamically rather than statically.

From reading through boost/asio/read.hpp and boost/asio/write.hpp once could implement an object with an interface that supported the expected API of read_some, async_read_some, write_some, async_write_some. However, it still leaves us stuck with templates which force the templatization of types up to the topmost user apis as illustrated in my example above.

@oberstet any other insights here? I am leaning towards just leaving things templated for now until we have a more compelling need.

@davidchappelle I think the right transport abstraction would probably more send/receive whole messages. So essentially, what the spec talks about with that bidirectional, reliable message channel. A standard ASIO stream pair would be wrapped in a class we provide, which implements a transport interface that is message based. And then this class is given as a template parameter. Maybe something like this works?

I am also wondering if the transport should be a template parameter anyway, or not more sort of a list of transport things that implement the transport interface. Alllowing run-time polymorphism at least here would be neat.

Just getting back to this. I'll draft a commit based on the discussion above.

I think I understand your point better now about abstracting in terms of a message based transport. I should have something ready later today that is much more in line with what you are describing.

@davidchappelle awesome!
FWIW, eg in AutobahnPython, WAMP transports are abstracted via 2 interfaces:

ITransport (https://github.com/tavendo/AutobahnPython/blob/master/autobahn/wamp/interfaces.py#L123)
ITransportHandler (https://github.com/tavendo/AutobahnPython/blob/master/autobahn/wamp/interfaces.py#L171)

Serializers are created (or given to) concrete transports, but the abstract transport interface just shuffles WAMP messages in their "parsed" form (eg https://github.com/tavendo/AutobahnPython/blob/master/autobahn/wamp/message.py#L222)
In AutobahnJS, we also have a message-based transport abstraction, eg https://github.com/tavendo/AutobahnJS/blob/master/package/lib/transport/websocket.js#L45 and the sibling files in that folder.

Does it still make sense for session to take in an in/out transport? Seems that by abstracting a transport it can be up to the transport itself how to manage its in/out streams.

No, in hindsight, I think this is over-engineering. Also, the lifetimes of Sessions and Transports should be completely decoupled (the Session ctor does NOT take a Transport and vice versa) - another lesson learned. Let me think for some minutes .. will post here again.

I'd approach it like this:

define 2 pure abstract base classes ITransport and ITransportHandler
ITransport has virtual void send(const wamp_message&) = 0, virtual bool is_open() const = 0 and virtual void attach(ITransportHandler&) = 0
ITransportHandler has virtual void on_attach(ITransport&) = 0, virtual void on_message(const wamp_message&) = 0 and virtual void on_detach(bool was_clean, std::string close_reason) = 0

Then, Session implements ITransportHandler, and concrete Transports (eg WAMP-over-WebSocket using MsgPack) implement ITransport.
When the boilerplate has created and opened a transport, it attaches the transport to a session (which is created completely independent of the transport). The session in on_attach will only save a reference/pointer to the transport. Then, when calling session.join, the session will use ITransport to send messages, and get on_message called and process incoming messages.
When the transport looses it's connection, it will also call on_detach. But a session may also be detached from a transport without the transport being closed. etc etc
Before jumping into implementation, we should hash out above. The implementation is more or less straightforward. The hard part is getting above right.

More braindumping: in AB C++, WAMP messages are abstracted here https://github.com/tavendo/AutobahnCpp/blob/master/autobahn/wamp_message.hpp
This isn't really an abstraction of WAMP messages: it's just a array of msgpack objects. Not every array of msgpack objects is a valid WAMP message.
That is, right now, transport would be responsible for 1) framing streams of bytes into message frames and 2) parsing those according to serializer in use into msgpack object arrays.
But the further parsing of arrays of msgpack objects into real WAMP messages happens inside Session, and there isn't even classes or what for say HELLO or CALL (the WAMP messages).
With AB Py, transports are responsible for 1), 2) and 3)!
This is important for the following reason: when running an AB Py WAMP component "side-by-side" (inside CB), the "transport" used between the component and the router boils down to simple function calls. When 3) isn't done in the transport, the session will still need to run "parse" and check code before actually performing the action.
Note that I am not saying one or the other is strictly better, I just want to raise awareness to these aspects.
Unrelated sidenote: what is this zone stuff eg here https://github.com/tavendo/AutobahnCpp/blob/master/autobahn/wamp_session.ipp#L805?

I see where you are going with this @oberstet. This would allow us to cycle through a list of transports trying to establish a session as well as recycle a transport for use by multiple successive sessions. Should be pretty straight forward.
As for the zone stuff, msgpack uses a concept called a zone. When you create message pack objects that do not fit into the intrinsic type category, the object requires a zone in order to allocate memory. There is no way to deallocate using a zone though. So a zone cannot have an indefinite scope as it would just continue to grow unbounded. As a result, zones should only really live as long as the objects that were allocated from it.

I see where you are going ..

Exactly. These are all lessons (I think) I've learned;)
Further, the WAMP spec has a section about relation between WAMP session and transports. But this is unnec. limiting. E.g. when we get session freeze/resume, a session can certainly outlive a transport. So I'd vote for removing this from the spec altogether (btw: if you can shelve out at least a little time, pls respond on the Hybi list. please.) Reusing a transport with different sessions was already possible anyway (and AB Py exposes it, but not AB JS).

another note rgd ITransportHandler: I have named the callbacks on_attach and on_detach - NOT on_open and on_close. The latter would suggest the transport connection would have opened or closed. But that isn't the point. The transport might have been opened long ago and used for whatever, but then is attached to a transport consumer (ITransportHandler). Of course a transport can only be attached to 1 transport handler at a time .. but.

Hbyi?
On Thu, Oct 1, 2015 at 4:20 PM, Tobias Oberstein<EMAIL_ADDRESS>wrote:

I see where you are going ..
Exactly. These are all lessons (I think) I've learned;)
Further, the WAMP spec has a section about relation between WAMP session
and transports. But this is unnec. limiting. E.g. when we get session
freeze/resume, a session can certainly outlive a transport. So I'd vote for
removing this from the spec altogether (btw: if you can shelve out at least
a little time, pls respond on the Hybi list. please.) Reusing a transport
with different sessions was already possible anyway (and AB Py exposes it,
but not AB JS).
—
Reply to this email directly or view it on GitHub
https://github.com/tavendo/AutobahnCpp/issues/54#issuecomment-144836323.

@davidchappelle https://mailarchive.ietf.org/arch/search/?email_list=hybi .. the threads on WAMP. I have posted a draft RFC (https://tools.ietf.org/html/draft-oberstet-hybi-tavendo-wamp-00) - but unless we can show significant interest by multiple parties in developing this RFC, the Hybi/IETF chairs will likely reject it soon! I perfectly understand that you have tight time budget (me too), but this is a critical phase with WAMP. And there is a window of opportunity - which will close again (because the Hybi/IETF chairs want to close the WG).

Good discussion here, I like the direction that this is (hopefully) going.
I was just going to say that I don't see a strong argument to introduce two interfaces where one might be sufficient - what you call ITransport is the thing that needs to be abstract and will be the base of various implementations, but ITransportHandler is apparently just the session and nothing else. Why do we want to make an interface for the session when (a) ITransport targets WAMP specifically and (b) the session is the only attachment target?
Also, maybe just to clarify this point, I'm under the assumption that ITransport & Co. are just working titles since they don't match too well with the current naming style in AutobahnCpp (which is more like STL/Boost and uses neither camel-casing nor I* prefixes). Correct?

Yes I am sticking to current naming conventions. The abstraction of
wamp_transport_handler is helpful to break the cyclic reference between
wamp_session and wamp_transport.
On Thu, Oct 1, 2015 at 5:26 PM, Jakob Petsovits<EMAIL_ADDRESS>wrote:

Good discussion here, I like the direction that this is (hopefully) going.
I was just going to say that I don't see a strong argument to introduce
two interfaces where one might be sufficient - what you call ITransport
is the thing that needs to be abstract and will be the base of various
implementations, but ITransportHandler is apparently just the session and
nothing else. Why do we want to make an interface for the session when (a)
ITransport targets WAMP specifically and (b) the session is the only
attachment target?
Also, maybe just to clarify this point, I'm under the assumption that
ITransport & Co. are just working titles since they don't match too well
with the current naming style in AutobahnCpp (which is more like STL/Boost
and uses neither camel-casing nor I* prefixes). Correct?
—
Reply to this email directly or view it on GitHub
https://github.com/tavendo/AutobahnCpp/issues/54#issuecomment-144855079.

FWIW, this should make adding support for native transports much simpler
and improve performance since full serialization/copy will not be required.
On Thu, Oct 1, 2015 at 7:37 PM, David Chappelle<EMAIL_ADDRESS>wrote:

Yes I am sticking to current naming conventions. The abstraction of
wamp_transport_handler is helpful to break the cyclic reference between
wamp_session and wamp_transport.
On Thu, Oct 1, 2015 at 5:26 PM, Jakob Petsovits<EMAIL_ADDRESS>wrote:

Good discussion here, I like the direction that this is (hopefully) going.
I was just going to say that I don't see a strong argument to introduce
two interfaces where one might be sufficient - what you call ITransport
is the thing that needs to be abstract and will be the base of various
implementations, but ITransportHandler is apparently just the session
and nothing else. Why do we want to make an interface for the session when
(a) ITransport targets WAMP specifically and (b) the session is the only
attachment target?
Also, maybe just to clarify this point, I'm under the assumption that
ITransport & Co. are just working titles since they don't match too well
with the current naming style in AutobahnCpp (which is more like STL/Boost
and uses neither camel-casing nor I* prefixes). Correct?
—
Reply to this email directly or view it on GitHub
https://github.com/tavendo/AutobahnCpp/issues/54#issuecomment-144855079
.

I have some questions around implementation details for detaching transport handlers. It seems that ITransport also needs to have a method to detach the ITransportHandler:
void ITransport::detach(bool was_clean, const std::string& reason)

What is the transport handlers implementation supposed to do when on_detach is called? If was_clean is true should it try and leave the session? What about if was_clean is false?

I took a look at AB Py for some clues but it looks like is using the old API on_open and on_close.

Could you create a 1st PR with only the 2 abstract bases for the interfaces and nothing else? We can describe the semantics of the interface in the docstrings of the respective methods.  And discuss the semantics of on_detach and such based on a concrete proposal. When we've done that, the implementation will fall into place I guess. Sounds good?

Pull request submitted as PR #58.
On a side note, I can't find a way to post anything to that thread on Hybi.
Seems to require a username/password to login which I don't have and can't
figure out how to create one.
On Fri, Oct 2, 2015 at 12:51 PM, Tobias Oberstein<EMAIL_ADDRESS>wrote:

Could you create a 1st PR with only the 2 abstract bases for the
interfaces and nothing else? We can describe the semantics of the interface
in the docstrings of the respective methods. And discuss the semantics of
on_detach and such based on a concrete proposal. When we've done that, the
implementation will fall into place I guess. Sounds good?
—
Reply to this email directly or view it on GitHub
https://github.com/tavendo/AutobahnCpp/issues/54#issuecomment-145084447.

To participate in the hypi mailing list, you need to be a subscriber. Subscription is quick and painless - https://www.ietf.org/mailman/listinfo/hybi

@davidchappelle I merged PR #58 and now do some changes. Let's continue the discussion here ..
Rgd. the Hybi list: does it work for you now?

Regarding wamp_transport_handler.on_message and wamp_transport.send: the one question we need to resolve is wamp_message (and hence is transparent to the interface definition formally, but still): will this be a list of msgpack objects (as it is right now), or will it be a proper, full parsed WAMP message abstraction?

The issue 2) is about flow-control:
Consider a session that wants to issue calls to a procedure "as fast as possible".
A simple loop that will issue calls without waiting for the previous call to return might overwhelm the transport or the callee.
A loop that will issue calls one after the other (that is, only issue a new call when the last has returned) won't overwhelm the transport or callee, but might run slower than possible.

@oberstet I have made wamp_message a simple class for the time being. It
simply wraps a vector of msgpack objects and a zone for the time being.
Eventually we could do what I did for bonefish and make each wamp message
type a fully parsed concrete type but what I have now should suffice for
the time being until we get the current abstraction work finished.
Flow control should be easy to manage by the user by simply waiting for
each future to resolve before issuing the next request. I think this
provides the most amount of flexibility.
I am in the process of updating all of the wamp message processing in the
session class to now work with the message class.
On Mon, Oct 5, 2015 at 9:03 AM, Tobias Oberstein<EMAIL_ADDRESS>wrote:

The issue 2) is about flow-control:
Consider a session that wants to issue calls to a procedure "as fast as
possible".
A simple loop that will issue calls without waiting for the previous call
to return might overwhelm the transport or the callee.
A loop that will issue calls one after the other (that is, only issue a
new call when the last has returned) won't overwhelm the transport or
callee, but might run slower than possible.
—
Reply to this email directly or view it on GitHub
https://github.com/tavendo/AutobahnCpp/issues/54#issuecomment-145521363.

@davidchappelle I agree the problem 1) can be postponed.
We should get the wamp_transport.send and wamp_transport_handler.on_message right first. This is a tricky one.
Currently, we use boost::asio::write under the hood. If a transport is using this to implement wamp_transport.send, a slow receiver which excerts TCP backpressure will simply make the program halt, since the socket write call is blocked in the kernel when the outgoing TCP buffers get full. IOW: wamp_transport.send might block, which is bad of course.
FWIW, the solution that Twisted implements is: a write never blocks. The data is simply buffered in user space. Of course the question then is: how to prevent memory exhaustion. The Twisted answer to this is Twisted Producer/Consumers.

simply waiting for each future to resolve

Which future? wamp_transport.send returns void ..

The user must eventually wait on futures returned by the session API for
things like calling/publishing/etc to complete. For most applications this
may be sufficient for achieving flow control.
Yes any tcp backpressure would be potentially problematic. In the current
implementation since we post all message to the io service in the session
object, the io service would basically act as a buffering mechanism. The
write itself is done in the context of the io_service. I think that
selecting a better mechanism for queuing writes may be able to be done as
an enhancement?
On Mon, Oct 5, 2015 at 9:28 AM, Tobias Oberstein<EMAIL_ADDRESS>wrote:

simply waiting for each future to resolve
Which future? wamp_transport.send returns void ..
—
Reply to this email directly or view it on GitHub
https://github.com/tavendo/AutobahnCpp/issues/54#issuecomment-145527333.

Publish is unacknowledge by default, so there is no future at the session level to wait on.
FWIW, here is a sketch of how the Twisted way would roughly translate:
class wamp_transport
{
public:
    /*!
     * Send the given WAMP message over the transport to the peer.
     *
     * This method does _not_ block. An implementation of this method will
     * usually serialize the WAMP message to a byte string, frame the bytes
     * according to the WAMP transport framing in place (e.g. WebSocket or
     * RawSocket) and then buffer and arrange for asynchronous sending of
     * the bytes, but return immediately.
     *
     * Note that when you first send message 1 and then message 2, _and_
     * message 2 was indeed received by the peer, it is guaranteed that
     * a) message 1 was also received, and b) that the order of receiving
     * first message 1 and then 2 is preserved. Hence a transport guarantees
     * strict message ordering and a slightly weaker form of reliability.
     *
     * @param message The message to be sent.
     */
    virtual void send(const std::shared_ptr<wamp_message>& message) = 0;

    /*!
     * Pause receiving messages on the attached transport handler. This is
     * used by transport handlers to excert backpressure on the sending peer.
     */
    virtual void pause() = 0;

    /*!
     * Resume receiving messages on the attached transport handler. This is
     * used by transport handlers to signal readiness to consume more messages
     * and lift backpressure from the sending peer.
     */
    virtual void resume() = 0;
};

class wamp_transport_handler
{
public:
    /*!
     * Called by the transport when a message is received.
     *
     * @param message The message that has been received.
     */
    virtual void on_message(const wamp_message& message) = 0;

    /*!
     * Called by the transport when outgoing buffers are filling up (have reached
     * the high watermark of buffered data) and this transport handler should (temporarily)
     * stop sending messages.
     */
    virtual void on_pause() = 0;

    /*!
     * Called by the transport when outgoing buffers are getting empty (have reached
     * the low watermark of buffered data) and this transport handler may resume
     * sending messages.
     */
    virtual void on_resume() = 0;
};

I like that API. what would be the behaviour of the session object when
on_pause() is called?
On Mon, Oct 5, 2015 at 9:44 AM, Tobias Oberstein<EMAIL_ADDRESS>wrote:

Publish is unacknowledge by default, so there is no future at the session
level to wait on.
FWIW, here is a sketch of how the Twisted way would roughly translate:
class wamp_transport
{public:
/*!     * Send the given WAMP message over the transport to the peer.     *     * This method does not block. An implementation of this method will     * usually serialize the WAMP message to a byte string, frame the bytes     * according to the WAMP transport framing in place (e.g. WebSocket or     * RawSocket) and then buffer and arrange for asynchronous sending of     * the bytes, but return immediately.     *     * Note that when you first send message 1 and then message 2, and     * message 2 was indeed received by the peer, it is guaranteed that     * a) message 1 was also received, and b) that the order of receiving     * first message 1 and then 2 is preserved. Hence a transport guarantees     * strict message ordering and a slightly weaker form of reliability.     *     * @param message The message to be sent.     */
virtual void send(const std::shared_ptr<wamp_message>& message) = 0;
/*!     * Pause receiving messages on the attached transport handler. This is     * used by transport handlers to excert backpressure on the sending peer.     */
virtual void pause() = 0;

/*!     * Resume receiving messages on the attached transport handler. This is     * used by transport handlers to signal readiness to consume more messages     * and lift backpressure from the sending peer.     */
virtual void resume() = 0;

};
class wamp_transport_handler
{public:
/*!     * Called by the transport when a message is received.     *     * @param message The message that has been received.     */
virtual void on_message(const wamp_message& message) = 0;
/*!     * Called by the transport when outgoing buffers are filling up (have reached     * the high watermark of buffered data) and this transport handler should (temporarily)     * stop sending messages.     */
virtual void on_pause() = 0;

/*!     * Called by the transport when outgoing buffers are getting empty (have reached     * the low watermark of buffered data) and this transport handler may resume     * sending messages.     */
virtual void on_resume() = 0;

};
—
Reply to this email directly or view it on GitHub
https://github.com/tavendo/AutobahnCpp/issues/54#issuecomment-145531605.

When on_pause was called, the session goes into "paused" state. In "paused" state, any call to session.call, session.publish, session.register and session.subscribe will raise a transport_paused exception.

Here is an alternative design:
class wamp_transport
{
public:
    /*!
     * Send the given WAMP message over the transport to the peer.
     *
     * The transport will resolve the returned future as soon as it is able
     * and willing to send more messages. This mechanism is used by the
     * transport to excert backpressure on the transport handler from the
     * receiving peer.
     *
     * @param message The message to be sent.
     */
    virtual boost::future<void> send(const std::shared_ptr<wamp_message>& message) = 0;
};

class wamp_transport_handler
{
public:
    /*!
     * Called by the transport when a message is received.
     *
     * The transport handler should resolve the returned future as soon as it is
     * able and willing to consume more messages. This mechanism can be used by
     * the transport handler to excert backpressure on the sending peer.
     *
     * @param message The message that has been received.
     */
    virtual boost::future<void> on_message(const wamp_message& message) = 0;
};

The downside with this approach is that it introduces a lot of additional
locking overhead for each message sent.
On Mon, Oct 5, 2015 at 10:01 AM, Tobias Oberstein<EMAIL_ADDRESS>wrote:

Here is an alternative design:
class wamp_transport
{public:
/*!     * Send the given WAMP message over the transport to the peer.     *     * The transport will resolve the returned future as soon as it is able     * and willing to send more messages. This mechanism is used by the     * transport to excert backpressure on the transport handler from the     * receiving peer.     *     * @param message The message to be sent.     /
virtual boost::future send(const std::shared_ptr<wamp_message>& message) = 0;
};
class wamp_transport_handler
{public:
/!     * Called by the transport when a message is received.     *     * The transport handler should resolve the returned future as soon as it is     * able and willing to consume more messages. This mechanism can be used by     * the transport handler to excert backpressure on the sending peer.     *     * @param message The message that has been received.     */
virtual boost::future on_message(const wamp_message& message) = 0;
};
—
Reply to this email directly or view it on GitHub
https://github.com/tavendo/AutobahnCpp/issues/54#issuecomment-145536495.

Locking? Why? This whole stuff should be single-threaded anyway?

Here is another (somewhat orthogonal) Q we need to decide:
a) have wamp_transport and wamp_transport_handler OR
b) only have wamp_transport and ..
let wamp_transport provide a listener pattern, so I can hook 1 or more listeners for: on_message, on_pause etc

Futures are implemented with mutexes and condition variables. So using them
results in a bunch of locking overhead even if you are in a single threaded
context. Albeit an uncontended mutex will probably benefit from futex type
behaviour.
Not sure if multiple listeners would be very useful. If that type of
behaviour is desired it could be left up to the user of the session api to
fan out to multiple listeners.
On Mon, Oct 5, 2015 at 10:08 AM, Tobias Oberstein<EMAIL_ADDRESS>wrote:

Locking? Why? This whole stuff should be single-threaded anyway?
Here is another (somewhat orthogonal) Q we need to decide:
a) have wamp_transport and wamp_transport_handler OR
b) only have wamp_transport and ..
let wamp_transport provide a listener pattern, so I can hook 1 or more
listeners for: on_message, on_pause etc
—
Reply to this email directly or view it on GitHub
https://github.com/tavendo/AutobahnCpp/issues/54#issuecomment-145539153.

I think that we should we break this work up into multiple deliverables as
the code changes are starting to get to be fairly significant and we can
deliver it in smaller incremental changes.

Wamp transport abstraction

Implement core abstraction for wamp transports to decouple rawsocket
transports from sessions.
This would NOT include on_attach and on_detach behaviour just the
simple concept of transports.

Implement transport decoupling

Implement on_attach and on_detach behaviour for
wamp_transport_handler as well as the new proposed API for initializing
transports/sessions (i.e. setup or main style).
This will eliminate some boiler plate and allow for transport and session
reuse.

Implement flow control for transports

Implement the on_suspend and on_resume for wamp_transport and
wamp_transport_handler.
This would provide some flow control so that the session cannot be
overwhelmed with requests.

On Mon, Oct 5, 2015 at 10:15 AM, David Chappelle<EMAIL_ADDRESS>wrote:

Futures are implemented with mutexes and condition variables. So using
them results in a bunch of locking overhead even if you are in a single
threaded context. Albeit an uncontended mutex will probably benefit from
futex type behaviour.
Not sure if multiple listeners would be very useful. If that type of
behaviour is desired it could be left up to the user of the session api to
fan out to multiple listeners.
On Mon, Oct 5, 2015 at 10:08 AM, Tobias Oberstein <
<EMAIL_ADDRESS>wrote:

Locking? Why? This whole stuff should be single-threaded anyway?
Here is another (somewhat orthogonal) Q we need to decide:
a) have wamp_transport and wamp_transport_handler OR
b) only have wamp_transport and ..
let wamp_transport provide a listener pattern, so I can hook 1 or more
listeners for: on_message, on_pause etc
—
Reply to this email directly or view it on GitHub
https://github.com/tavendo/AutobahnCpp/issues/54#issuecomment-145539153
.

I think I disagree on this one: IMO we should first define the proper interface.
The implementation can lack behind the interface, but the interface should be designed and fixed now. I'd prefer a half-baked implementation of a good and stable interface over an implementation of a half-baked interface.

the code changes are starting to get to be fairly significant

What are you implementing? The interface isn't "done", so there isn't anything to implement (or you are chasing a moving target) - maybe we have different modes of working;)

There is a bunch of work to be done regardless of the interface(s) so I
have been tackling that between discussions. I have also created
wamp_rawsocket_transport and extracted all of the wamp_session related
code into that class. Basically all of the message framing/receiving etc.
Most of the interface that we have talked about at this point is just
stubbed out for the time being. Might be easier to get my refactoring in at
a minimum and then focus on implementing the API internals.
On Mon, Oct 5, 2015 at 12:51 PM, Tobias Oberstein<EMAIL_ADDRESS>wrote:

I think I disagree on this one: IMO we should first define the proper
interface.
The implementation can lack behind the interface, but the interface should
be designed and fixed now. I'd prefer a half-baked implementation of a good
and stable interface over an implementation of a half-baked interface.
the code changes are starting to get to be fairly significant
What are you implementing? The interface isn't "done", so there isn't
anything to implement (or you are chasing a moving target) - maybe we have
different modes of working;)
—
Reply to this email directly or view it on GitHub
https://github.com/tavendo/AutobahnCpp/issues/54#issuecomment-145596545.

b) only have wamp_transport and ..

class wamp_transport
{
public:
    /*
     * SENDER interface
     */

    /*!
     * Send the given WAMP message over the transport to the peer.
     *
     * @param message The message to be sent.
     */
    virtual void send_message(const std::shared_ptr<wamp_message>& message) = 0;

    /*!
     * Listener type for "on_pause" events.
     */
    typedef std::function<void()> on_pause_t;

    /*!
     * Add a listener to "on_pause" events.
     */
    virtual void on_pause(on_pause_t& listener) = 0;

    /*!
     * Listener type for "on_resume" events.
     */
    typedef std::function<void()> on_resume_t;

    /*!
     * Add a listener to "on_resume" events.
     */
    virtual void on_resume(on_resume_t& listener) = 0;

    /*
     * RECEIVER interface
     */

    /*!
     * Listener type for "on_message" events.
     */
    typedef std::function<void(const wamp_message&)> on_message_t;

    /*!
     * Add a listener to "on_message" events.
     */
    virtual void on_message(on_message_t& listener) = 0;

    /*!
     * Pause receiving messages and firing "on_message" events. This is
     * used to excert backpressure on the sending peer.
     */
    virtual void pause() = 0;

    /*!
     * Resume receiving messages and firing "on_message" events. This is
     * used to signal readiness to consume more messages and lift backpressure
     * from the sending peer.
     */
    virtual void resume() = 0;
};

So the on_message handler here is replacing
transport_handler::on_message?
On Mon, Oct 5, 2015 at 2:52 PM, Tobias Oberstein<EMAIL_ADDRESS>wrote:

b) only have wamp_transport and ..
class wamp_transport
{public:
/*     * SENDER interface     */
/*!     * Send the given WAMP message over the transport to the peer.     *     * @param message The message to be sent.     */
virtual void send_message(const std::shared_ptr<wamp_message>& message) = 0;

/*!     * Listener type for "on_pause" events.     */
typedef std::function<void()> on_pause_t;

/*!     * Add a listener to "on_pause" events.     */
virtual void on_pause(on_pause_t& listener) = 0;

/*!     * Listener type for "on_resume" events.     */
typedef std::function<void()> on_resume_t;

/*!     * Add a listener to "on_resume" events.     */
virtual void on_resume(on_resume_t& listener) = 0;

/*     * RECEIVER interface     */

/*!     * Listener type for "on_message" events.     */
typedef std::function<void(const wamp_message&)> on_message_t;

/*!     * Add a listener to "on_message" events.     */
virtual void on_message(on_message_t& listener) = 0;

/*!     * Pause receiving messages and firing "on_message" events. This is     * used to excert backpressure on the sending peer.     */
virtual void pause() = 0;

/*!     * Resume receiving messages and firing "on_message" events. This is     * used to signal readiness to consume more messages and lift backpressure     * from the sending peer.     */
virtual void resume() = 0;

};
—
Reply to this email directly or view it on GitHub
https://github.com/tavendo/AutobahnCpp/issues/54#issuecomment-145630610.

Yeah. This proposal is to further illustrate the design space: doing away with transport_handler altogether.
I am still exploring possibilities.
Above last proposal doesn't work quite that way: C++ makes it hard to have pointers to methods on an object (the implicit "this").
e.g. virtual void on_message(on_message_t& listener) = 0; does not work with a possible wamp_session::on_message implementation (I admit, I am spoiled by Python).

fwiw, here is the last proposal with slightly better/readable naming:

https://github.com/tavendo/AutobahnCpp/blob/transport_ifc/autobahn/wamp_transport.hpp#L41

And here is my current sketch of retaining 2 interfaces:

https://github.com/tavendo/AutobahnCpp/blob/transport_ifc/autobahn/wamp_transport2.hpp
https://github.com/tavendo/AutobahnCpp/blob/transport_ifc/autobahn/wamp_transport_handler.hpp

one last note (need some sleep): I have thought about why would one need flow-control at the WAMP transport level (when with most WAMP actions, there is feedback at the WAMP level .. call result etc on which one could hook on flow control).
Here is my list:

unacknowledged publications
progressive results
progressive calls (not yet there, but we'll have it)

Plus: 4. throttling calls based on call result won't run at maximum speed (since the roundtrip latency isn't hidden and used for actual work).
Of these, I think 2 and 3 are of real practical use (because the whole point of the "progressive" stuff is about sending/receiving larger amounts of data at the WAMP level).

can be circumvented using acknowledged publications.

And 4. is probably a minor thing .. but still.

Should be possible with std::bind. For example:
m_transport.set_message_handler(std::bind(&wamp_session::on_message, this,
std::placeholders::_1));
On Mon, Oct 5, 2015 at 6:14 PM, Tobias Oberstein<EMAIL_ADDRESS>wrote:

Yeah. This proposal is to further illustrate the design space: doing away
with transport_handler altogether.
I am still exploring possibilities.
Above last proposal doesn't work quite that way: C++ makes it hard to have
pointers to methods on an object (the implicit "this").
e.g. virtual void on_message(on_message_t& listener) = 0; does not work
with a possible wamp_session::on_message implementation (I admit, I am
spoiled by Python).
—
Reply to this email directly or view it on GitHub
https://github.com/tavendo/AutobahnCpp/issues/54#issuecomment-145685115.

is less than ideal as you point out, the pipe will not be full and
latency accumulation will be very limiting for large amounts of data.

I am wondering if flow control can be an optional feature. For simple
applications it is likely insignificant and having the odd write block may
not be of any real concern. Or for cases where latency is a non issue and
throughput is high like loopback, unix domain sockets, and native
communication.
On Mon, Oct 5, 2015 at 7:42 PM, David Chappelle<EMAIL_ADDRESS>wrote:

Should be possible with std::bind. For example:
m_transport.set_message_handler(std::bind(&wamp_session::on_message, this,
std::placeholders::_1));
On Mon, Oct 5, 2015 at 6:14 PM, Tobias Oberstein<EMAIL_ADDRESS>
wrote:

Yeah. This proposal is to further illustrate the design space: doing away
with transport_handler altogether.
I am still exploring possibilities.
Above last proposal doesn't work quite that way: C++ makes it hard to
have pointers to methods on an object (the implicit "this").
e.g. virtual void on_message(on_message_t& listener) = 0; does not work
with a possible wamp_session::on_message implementation (I admit, I am
spoiled by Python).
—
Reply to this email directly or view it on GitHub
https://github.com/tavendo/AutobahnCpp/issues/54#issuecomment-145685115
.

I am going to cleanup what I have so far and push to my fork. Then you can
see what I have done to cleanup messages and decoupling transports from
sessions. Once we have settled on an API it should be very simple to
incorporate those api changes.
On Mon, Oct 5, 2015 at 7:54 PM, David Chappelle<EMAIL_ADDRESS>wrote:

is less than ideal as you point out, the pipe will not be full and
latency accumulation will be very limiting for large amounts of data.

I am wondering if flow control can be an optional feature. For simple
applications it is likely insignificant and having the odd write block may
not be of any real concern. Or for cases where latency is a non issue and
throughput is high like loopback, unix domain sockets, and native
communication.
On Mon, Oct 5, 2015 at 7:42 PM, David Chappelle<EMAIL_ADDRESS>wrote:

Should be possible with std::bind. For example:
m_transport.set_message_handler(std::bind(&wamp_session::on_message,
this, std::placeholders::_1));
On Mon, Oct 5, 2015 at 6:14 PM, Tobias Oberstein <
<EMAIL_ADDRESS>wrote:

Yeah. This proposal is to further illustrate the design space: doing
away with transport_handler altogether.
I am still exploring possibilities.
Above last proposal doesn't work quite that way: C++ makes it hard to
have pointers to methods on an object (the implicit "this").
e.g. virtual void on_message(on_message_t& listener) = 0; does not work
with a possible wamp_session::on_message implementation (I admit, I am
spoiled by Python).
—
Reply to this email directly or view it on GitHub
https://github.com/tavendo/AutobahnCpp/issues/54#issuecomment-145685115
.

@davidchappelle Thanks for the PR! It's unfortunate that I can commit more time to this right now .. I hope I can get some high-prio blockers out of the way soon, and the dive into C++ again.
Sidenote: we've published a new Internet-Draft: https://tools.ietf.org/html/draft-oberstet-hybi-tavendo-wamp-02
This now contains both the basic and the advanced profile. Hashing out this RFC is one of the high-prios on my list. Some of the features in Crossbar.io have surpassed the spec, and I want to bring those back in line ..

How would you like to proceed on the wamp transport front? The transport API is how we had discussed and amended above. I apologize for the large commit but once you swap out the transport a number of other changes need to be made. So there shouldn't be many surprises there. For example, the wamp message. We needed to change the message structure a bit which resulted in a bunch of corresponding changes in all of the message marshalling/unmarshalling. This is one of the reasons why the commit looks large. There are lots of incidental changes like this that needed to be made. FWIW, we are using what I currently have on my fork davidchappelle/feature/wamp_transport_abstraction quite extensively in a production environment.

I am going to decompose my commit into several smaller commits that will be easier to review.


⚠ Processors has degraded performance

In feb1288, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 49 ms

Resolved: Processors performance has improved in b31bcde.


feat(exec): add new public funcion OutputWithErr

will also return an error, if any occurs.

We want to keep the public API minimal and stable. In this case we cater to the most common use-case. If you really think this will be useful across the whole ecosystem I say let's merge, otherwise I would probably say that write this func in the repo you want to use it :)
We actually started out with this setup in the beginning but actively chose to keep it minimal.

Gotcha! Yeah I rather have the error returned and let the user choose wether to panic or not.

We want to keep the public API minimal and stable. In this case we cater to the most common use-case. If you really think this will be useful across the whole ecosystem I say let's merge, otherwise I would probably say that write this func in the repo you want to use it :)
We actually started out with this setup in the beginning but actively chose to keep it minimal.

Gotcha! Yeah I rather have the error returned and let the user choose wether to panic or not.

I would probably argue that you can write this in your repo without almost any overhead. I.e. copy the implementation of Output and remove the panic :)
It is basically a convenience func to avoid having to do an error check. The cmd.Output func is basically what you've implemented :)

Personally I find the Output function to almost not be needed here as it does very little compared to what you already get out of the Go stdlib and I'd rather get rid of it than introducing another one which also does very little.

Okidoki, ill just implement this locally then.


Distinguish between undetermined failures and API errors

Return 404 if project is not found. It is not an internal server error.
Introduce a separate error class webapi.APIError() to report them.
Fix #59: return no results form database wrapper if project is not found.
Add a single test case for #59. TODO: test more cases.
This is technically an API change since 404 is returned instead of 500 in some cases, @jpakkane please take a look.

Have you tried what happens in wraptool if this change is applied (I'm guessing nothing since it only cares about success or failure, not how it failed)? If things keep working then this LGTM.

$ ~/meson/meson.py wrap info sdb
Warning: SSL not available, traffic not authenticated.
Traceback (most recent call last):
  File "/home/user/meson/meson.py", line 29, in <module>
    sys.exit(mesonmain.main())
  File "/home/user/meson/mesonbuild/mesonmain.py", line 401, in main
    return run(sys.argv[1:], launcher)
  File "/home/user/meson/mesonbuild/mesonmain.py", line 320, in run
    return wraptool.run(remaining_args)
  File "/home/user/meson/mesonbuild/wrap/wraptool.py", line 224, in run
    info(args[0])
  File "/home/user/meson/mesonbuild/wrap/wraptool.py", line 139, in info
    jd = get_result(API_ROOT + 'projects/' + name)
  File "/home/user/meson/mesonbuild/wrap/wraptool.py", line 52, in get_result
    u = open_wrapdburl(urlstring)
  File "/home/user/meson/mesonbuild/wrap/wrap.py", line 70, in open_wrapdburl
    return urllib.request.urlopen(urlstring)
  File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: NOT FOUND

That is what happens. It throws an error anyway and I don't think it checks the error code:
$ git grep HTTPError
[empty result]


aws:efs:FileSystemPolicy keep wanting to update itself when the policy has conditions

Hi, when using aws:efs:FileSystemPolicy the resource keeps trying to update itself during pulumi up when the policy has conditions.
Expected behavior
Since there are no changes to the policy document itself, aws.efs.FileSystemPolicy shouldn't try to update.
Current behavior
aws.efs.FileSystemPolicy keeps trying to update itself, even if the policy document doesn't change.
    ~ aws:efs/fileSystemPolicy:FileSystemPolicy: (update)
        [id=fs-4a023e7f]
        [urn=urn:pulumi:prod::lytt::aws:efs/fileSystemPolicy:FileSystemPolicy::file_system_policy]
        [provider=urn:pulumi:prod::lytt::pulumi:providers:aws::default_3_32_0::de8b6716-24e9-431b-99ab-4aea511e3b8a]
      ~ policy: "{\n  \"Version\" : \"2012-10-17\",\n  \"Statement\" : [ {\n    \"Sid\" : \"\",\n    \"Effect\" : \"Allow\",\n    \"Principal\" : {\n      \"AWS\"
 : \"*\"\n    },\n    \"Action\" : \"elasticfilesystem:ClientMount\",\n    \"Resource\" : \"*\",\n    \"Condition\" : {\n      \"Bool\" : {\n        \"aws:SecureT
ransport\" : \"1\"\n      }\n    }\n  } ]\n}" => "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Sid\": \"\",\n      \"Effect\": \"Allow\",
\n      \"Action\": \"elasticfilesystem:ClientMount\",\n      \"Resource\": \"*\",\n      \"Principal\": {\n        \"AWS\": \"*\"\n      },\n      \"Condition\":
 {\n        \"Bool\": {\n          \"aws:SecureTransport\": [\n            \"1\"\n          ]\n        }\n      }\n    }\n  ]\n}"

Steps to reproduce
Here's a Python example:
import pulumi
import pulumi_aws as aws

efs = aws.efs.FileSystem("efs")

access_point = aws.efs.AccessPoint("acess_point",
                                   file_system_id=efs)

policy_document = aws.iam.get_policy_document(statements=[
    {
        "principals": [{
            "type": "AWS",
            "identifiers": ["*"]}],
        "actions": ["elasticfilesystem:ClientMount"],
        "resources": ["*"],
        "conditions": [
            {
                "test": "Bool",
                "values": [True],
                "variable": "aws:SecureTransport"}]}])

aws.efs.FileSystemPolicy("file_system_policy",
                         file_system_id=efs,
                         policy=policy_document.json)

More information
In this particular scenario I notice that the difference happens when there are conditions. The policy shows that, on the conditions, values is a list, but what gets saved is a single value, thus triggering the difference. Here's the diff:
14c14,16
<           "aws:SecureTransport": "1"
---
>           "aws:SecureTransport": [
>             "1"
>           ]

Using aws.iam.get_policy_document, so values must be a list.

Looks like a bug in the provider. You may be able to manually workaround by tweaking the policy json before passing it as an input.

Closing, I can't reproduce this any more with pulumi-aws 5.x.


Build failed with verbose output using aiken build -t verbose command

I attempted to trace the output value using trace assets.flatten(out.value) but encountered an error.
   aiken::fatal::error
   Whoops! You found a bug in the Aiken compiler.

   Please report this error at https://github.com/aiken-lang/aiken/issues/new.
   In your bug report please provide the information below and if possible the code
   that produced it.

   Operating System: macos
   Architecture:     aarch64
   Version:          v1.1.3+3d77b5c

   crates/aiken-lang/src/gen_uplc.rs:247:17

To reproduce the error:
if cond {
  trace assets.flatten(out.value)
  expect ...
}

Can we see more of the code you used to produce this error. The example given is not enough to reproduce it.

But I have a very strong feeling the error is caused by some code example that should result in a parse error.


Allow user to remove stale diagnostics

If the user modifies source and SourceKit diagnostics differ, they may want to cleanup all the old swiftc ones

Is there any situation where someone would want to keep the stale diagnostics?

Is there any situation where someone would want to keep the stale diagnostics?

I am concerned we are adding too many configuration settings. We now have 10 possible diagnostic lists. In what situation would someone want to not remove stale swifts diagnostics? Otherwise everything looks good

You're right, really just using it as a "kill switch" but I cannot think of a user case. Will address that

Hi @award999 ! I'm just catching up on this work and excited for the improvements. I thought VSCode did not have enough extensibility hooks to implement it. Do you know if this will work with non-SPM builds? We use CMake and sourcekit-lsp, and we use the $swiftc problem matcher on a CMake invocation.

@tristanlabelle this applies to swift tasks. The $swiftc problem matcher can still be used with any tasks you define, but no vscode does not allow for cleaning up diagnostics produced from a problem matcher, i.e. you need to run the task again


updating the existing pipeline CORS does not work

Summary
Modify pipeline CORS settings to * is not work.
Steps to reproduce

modify a pipeline which is not set the CORS, then leave the '*' and update pipeline
wait the pipeline status to be active
send the request by endpoint
the CORS error will appear with no Access-Control-Allow-Origin header

What is the current bug behavior?

What is the expected correct behavior?
Send the request success
Relevant logs and/or screenshots
Possible fixes
Create a new pipeline with CORS * is works, it seems to update the pipeline issue.

This is :bug: Bug Report

Ingestion server support updating CORS, the reason of the error is the pipeline version is too old, cannot support updating through control plane directly.
@tyyzqmf  please confirm it.

it's already fixed in a89d72e488


ImportError: cannot import name 'cpdGeneration_algorithm_optimized' from 'mmpdblib'

You should install the module 'mmpdblib' before you launch the code:
git clone https://github.com/mahendra-awale/medchem_moves
cd medchem_moves
python setup.py install

This step is shown in README.md:
https://github.com/illidanlab/MolSearch/blob/30f36f0a976b40ea52edad4d8f0ff1ef531cbf8a/README.md?plain=1#L20-L27

I came across the same problem. And I actually finished the steps below, but still encountered "ImportError: cannot import name 'cpdGeneration_algorithm_optimized' from 'mmpdblib' (/home/xyd/anaconda3/envs/mcts/lib/python3.7/site-packages/mmpdblib/init.py)"
 git clone https://github.com/mahendra-awale/medchem_moves
 cd medchem_moves
 python setup.py install


Prometheus not respecting Memory requests and limits.

What did you do?
I installed Prometheus via Helm-chart. Disabled a lot of extra options like alert-manager, policy-reports, grafana, etc. I only want to run Prometheus for Kiali. I even created a Dedicated Node with 64GB memory and 8 cores. Unfortunately, Prometheus keeps making insane Memory requests, like 150GB, 200GB.... That's causing the pod to get OOMKIlled.
Why is Prometheus requesting more memory than the limits already mentioned in helm-chart. If not, what's the point of adding limits overall.
Helm-chart config deployed via Argo:
grafana:
  enabled: false
alertmanager:
  enabled: false
prometheus-node-exporter:
  enabled: false
kube-state-metrics:
  enabled: false
prometheus-pushgateway:
  enabled: false
additionalServiceMonitors: []
podSecurityPolicy:
  enabled: false
server:
  enabled: true
  service:
    servicePort: 9090
    targetPort: 9090
    name: prometheus
    retention: "12h"
  persistentVolume:
    enabled: true
    size: 20Gi
  resources:
    limits:
      memory: "60Gi"
    requests:
      memory: "46Gi"
  defaultFlagsOverride:
    - "--storage.tsdb.retention.time=12h"
    - "--config.file=/etc/config/prometheus.yml"
    - "--storage.tsdb.path=/data"
    - "--web.console.libraries=/etc/prometheus/console_libraries"
    - "--web.console.templates=/etc/prometheus/consoles"
    - "--web.enable-lifecycle"
  tolerations:
    - key: "prometheus-only"
      operator: "Equal"
      value: "present"
      effect: "NoSchedule"
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: cloud.google.com/gke-nodepool
                operator: In
                values:
                  - prometheus-node-pool
  fullnameOverride: "prometheus"
  global:
    scrape_interval: 3m
    evaluation_interval: 3m
hostPort: null

What did you expect to see?
No response
What did you see instead? Under which circumstances?
Pod getting OOMKIlled
System information
Google GKE node
Prometheus version
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: prometheus
  namespace: istio-system
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: istio
  sources:
    - repoURL: https://prometheus-community.github.io/helm-charts
      chart: prometheus
      targetRevision: 25.27.0
      helm:
        releaseName: prometheus

Prometheus configuration file
No response
Alertmanager version
Disabled

Alertmanager configuration file
Disabled

Logs
Nothing different here. Pod is getting killed by Kubelet due to memory requests.

@crush-on-anechka explained the limit purpose correctly.
The 150-200GB of memory is pretty large, but possible if you are attempting to scrape hundreds millions of unique time series via a single Prometheus server.
I will mark it as answered. The ideas for Prometheus to do "more" to respect memory limit e.g.  drop majority of your metrics to fit into your limit are discussed here, feel free to join us to provide more ideas and the behaviour you would like to see!


how to create an admin (or privileged user)

Porting issue from aws-amplify
proper technique to create an admin (or privileged user)
https://github.com/aws/aws-amplify/issues/83

Roll Based Access Controls (rbac) using Cognito user groups would be nice
https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-user-groups.html
An example of  how an Admin within a user group could elevate privileges of a list of users for a given user group

tracking this feature here: https://github.com/aws/aws-amplify/issues/156


Updated version of httpoison

Hi All!
Newbie to the elixir world so sorry if this isn't right...
I recently added tentacat to my Phoenix project, running mix eps.get complained:
Looking up alternatives for conflicting requirements on httpoison
From mix.lock: 0.7.3
From tentacat v0.1.5: ~> 0.6.0
** (Mix) Hex dependency resolution failed, relax the version requirements or unlock dependencies
Updating the version of httpoison in tentacat fixed this.
MIX_ENV=test mix do deps.get, test
Runs fine.

Thanks!


try-puppeteer: require('puppeteer') not supported

https://try-puppeteer.appspot.com/ explains that it's not necessary to require('puppeteer') when pasting code, but it turns out it's not allowed, either. This makes it (slightly!) more difficult to copy and paste code.

This issue was moved to ebidel/try-puppeteer#8


Outhalo size warning

Tiny PR to fix #1466 .

I'm not sure I understand the fix. self._decomposition should have been already of the proper size, and so left and right, given that left = [.... in zip(self._decomposition, ...)]. Why do you need to further slice it?

I'm not sure I understand the fix. self._decomposition should have been already of the proper size, and so left and right, given that left = [.... in zip(self._decomposition, ...)]. Why do you need to further slice it?

left and right also contain the size of the time halo which we wan't to ignore for this spatial check.

Looks fine, any way to test it?

I was thinking about that, but came to the conclusion that a specific test was overkill. The incorrect warning currently (<- current master) shows up for the mpi seismic examples, but I'm not sure we wan't to play around with them or add a 'dummy' example that currently erronously produces the warning?

Thanks, merged.


Windows trojan warning

I try to install but it says there is trojan in the file. Is there a risk or is it false positive?

there is no virus.
u can download the source code, check and compile it by yourself if u dont trust releases

Answer above.


sd_setImageWithURL calls completion block twice when used with SDWebImageRefreshCached

Hi again,
Same code with SDWebImageRefreshCached, calls completion block twice.
[self.photoImageView sd_setImageWithURL:url
                       placeholderImage:nil
                                options:SDWebImageRetryFailed | SDWebImageRefreshCached
                               progress:^(NSInteger receivedSize, NSInteger expectedSize) {
                                   dispatch_async(dispatch_get_main_queue(), ^{
                                       NSLog(@"receivedSize = %d, expectedSize = %d", receivedSize, expectedSize);
                                   });
                               }
                              completed:^(UIImage *image, NSError *error, SDImageCacheType cacheType, NSURL *imageURL) {
                                  NSLog(@"Complete = %d, error = %@", cacheType, error);
                              }];

Log:
2015-03-23 12:38:09.112 App[50401:1875672] Complete = 2, error = (null)
2015-03-23 12:38:09.113 App[50401:1875672] receivedSize = 0, expectedSize = -1
2015-03-23 12:38:11.378 App[50401:1875672] receivedSize = 0, expectedSize = 34220
2015-03-23 12:38:11.378 App[50401:1875672] receivedSize = 1432, expectedSize = 34220
2015-03-23 12:38:11.378 App[50401:1875672] receivedSize = 2864, expectedSize = 34220
2015-03-23 12:38:11.378 App[50401:1875672] receivedSize = 4296, expectedSize = 34220
2015-03-23 12:38:11.380 App[50401:1875672] receivedSize = 5728, expectedSize = 34220
2015-03-23 12:38:11.380 App[50401:1875672] receivedSize = 7160, expectedSize = 34220
2015-03-23 12:38:11.380 App[50401:1875672] receivedSize = 8592, expectedSize = 34220
2015-03-23 12:38:11.381 App[50401:1875672] receivedSize = 10024, expectedSize = 34220
2015-03-23 12:38:11.381 App[50401:1875672] receivedSize = 11456, expectedSize = 34220
2015-03-23 12:38:11.381 App[50401:1875672] receivedSize = 12888, expectedSize = 34220
2015-03-23 12:38:11.572 App[50401:1875672] receivedSize = 14320, expectedSize = 34220
2015-03-23 12:38:11.572 App[50401:1875672] receivedSize = 15752, expectedSize = 34220
2015-03-23 12:38:11.583 App[50401:1875672] receivedSize = 18616, expectedSize = 34220
2015-03-23 12:38:11.597 App[50401:1875672] receivedSize = 21480, expectedSize = 34220
2015-03-23 12:38:11.598 App[50401:1875672] receivedSize = 22912, expectedSize = 34220
2015-03-23 12:38:11.598 App[50401:1875672] receivedSize = 24344, expectedSize = 34220
2015-03-23 12:38:11.599 App[50401:1875672] receivedSize = 25776, expectedSize = 34220
2015-03-23 12:38:11.599 App[50401:1875672] receivedSize = 27208, expectedSize = 34220
2015-03-23 12:38:11.599 App[50401:1875672] receivedSize = 28640, expectedSize = 34220
2015-03-23 12:38:11.599 App[50401:1875672] receivedSize = 30072, expectedSize = 34220
2015-03-23 12:38:11.601 App[50401:1875672] receivedSize = 32936, expectedSize = 34220
2015-03-23 12:38:11.602 App[50401:1875672] receivedSize = 34220, expectedSize = 34220
2015-03-23 12:38:11.618 App[50401:1875672] Complete = 0, error = (null)

Is this a desired behavior? It seems like a bug to me.

Update:
I removed second download request I had for the same UIImageView which happened earlier tho and now it's even worse, I get completion callback, then two updates, then silence.
2015-03-23 13:21:10.588 Sunset[52095:1907863] Complete = 2, error = (null)
2015-03-23 13:21:10.606 Sunset[52095:1907863] receivedSize = 0, expectedSize = -1
2015-03-23 13:21:10.606 Sunset[52095:1907863] receivedSize = 0, expectedSize = 34220
2015-03-23 13:21:10.606 Sunset[52095:1907863] receivedSize = 34220, expectedSize = 34220

the progress block depends on your server setup as well. if your server setup returns the size of each segment upload then you can use it to show progressing download with percentage.
try and download the same image with chrome. if you see the a progress showing size downloaded and size total then please share the link so we can debug it

I don't see the problem with download from Parse which I use. The progress seems legit. What does not seem legit is that progress is reported after Completion block or completion block is never called otherwise.

are you using the latest build?

Using SDWebImage (3.7.2) according to pod install

Sometimes it just calls completion block right away, reports that image is cached in memory and then reloads it from server because SDWebImageRefreshCached was set. But it does not call completion block after image is refresh from server.

i tried in the demo project and you are right on the fact that it does not call the completion block when
SDWebImageRefreshCached is set.
if you remove the SDWebImageRefreshCached it works fine and reports progress before calling the completion block

Thanks, yeah it seems odd, I think it should because that's the whole point of SDWebImageRefreshCached I guess..

from the comments inside the SDWebImagemanager.m
// If image was found in the cache & SDWebImageRefreshCached is provided, notify about the cached image AND try to re-download it in order to let a chance to NSURLCache to refresh it from server.

Yeah so even if I ignore cached image in completion handler, I still do not receive completion handler with refreshed image.

if there was a new image in the server then it should be downloaded.
After the 1st completion block, the one that notifies you that the image exists in cache, the manager tries to re-download it again from the web in order to give NSURLCache a chance to refresh it from server.
logically it all depends on the headers you supply back as a response.

I see, in my case it hits 304 for sure. I still think that this method defeats its purpose, e.g. too much of internal implementation details exposed, threading issues (#1089) and inconsistency that makes it totally useless in case if you want to animate anything alongside. The only way to get anything useful out of it is to use progress handler and completely forget about completion handler, which supposed to be called on completion, not randomly.

no worries. Productive criticism is welcome.
In your case since it hits 304, it does not re-download the image again as it has not been modified.
i am using it without any issues so far to show progress but i agree that there are bits and pieces that need to be addressed.
Please if you find some time and have anything to offer, submit a pull request and we will be happy to take a look at it.
Closing this.

@pronebird: I tried to fix issues you are complaining here(#1089 #1090 and some others)
You can fine it here:
https://github.com/timkoma/SDWebImage/tree/SDWebImageOperation%2BCanceling/SDWebImage
I think that it will call completedBlock with cancelled code in case '304', but it can be easy fixed.
I will be happy if some body can test it.:-)

@mythodeia thanks for discussing that matter with me.
@timkoma did you send PR already? I'll take a look at your fix.

https://github.com/rs/SDWebImage/pull/1069

@pronebird Did you have a time for review or test ?

@timkoma I've been working a lot on other projects and I had no time to deal with that bug so I approached the problem differently.


Local languages not supported

Steps to reproduce
Create a script with a function and a .EXAMPLE comment.
Run New-MarkdownHelp
Expected behavior
A generated markdown file with
## EXAMPLES

### EXAMPLE 1

Actual behavior
A generated markdown file with
## EXAMPLES

### BEISPIEL 1

Environment data
Windows 10
platyPs v0.12.0
PSVersion 5.1.17134.407

Reopening this issue to track the feature request for localization support.


Add primitive to deal with EAGAIN and O_NONBLOCK values on Linux and OSX

EAGAIN needs to return 11 on Linux and 35 on OSX and FreeBSD
O_NONBLOCK needs to return 2048 on Linux and 4 on OSX and FreeBSD

@jemc i'm going to merge this as it fixes a critical bug and @lispmeister is now away and then fix the formatting.


Support CentOS installation

Hi,
just trying to install on CentOS:
curl -sL https://ibm.biz/idt-installer | bash
[main] --==[ IBM Cloud Developer Tools for Linux/MacOS - Installer, v1.2.3 ]==--
[main] WARN: Linux has only been tested on Ubuntu, please let us know if you use this utility on other Distros
[install] Starting Full Installation...
[install_deps] Checking for and updating 'apt-get' support on Linux
which: no apt-get in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
[install_deps] ERROR: 'apt-get' is not found.  Thats the only linux installer I know, sorry.
Is there a way to install on Centos ?
Thank you

Hi @matteofcorona - Currently, we've only coded with apt-get based distros.  It will be a while before I can load up, code and test for CentOS, but I'll put it on my Todo list.   I wouldn't expect many roadblocks to make this work.  Google shows there are ways to install apt on centos, but this could lead to an ugly untested rabbit hole, but if you want to explore....
PRs / hints, guidance are always welcomed too!
You should be able to manually install all the pieces yourself.

install dependencies: curl, git, docker, kubectl, helm
Install IBM Cloud CLI:   https://console.bluemix.net/docs/cli/index.html
Install required "bx" plugins:  cloud-functions, container-registry, container-service, dev, schematics, sdk-gen

bx plugin install <plugin>

Optional: create launcher idt shell, which simply calls bx dev $*.   See towards the end of the installer where this is defined (Line ~377)

If you do these steps above, please take notes, as it will help me extend the real installer to support CentOS.  Also, if you run into other issues, let me know and I'll try to assist.  cheers.

Google shows there are ways to install apt on centos, but this could lead to an ugly untested rabbit hole, but if you want to explore....

I think yum would be an easier way to go, it should just be a case of if ubuntu|debian then apt install foo; elif centos|rhel then yum install foo.
@a-roberts would probably be a good guinea-pig for this as well (as a RHEL user).

Waiting for RHEL / CentOS version as well ...

Your support for only Ubuntu is making life difficult for IBM Cloud Private - ICP Knowledge Center link https://www.ibm.com/support/knowledgecenter/SSBS6K_<IP_ADDRESS>/manage_cluster/install_cli.html redirects for bx install to https://console.bluemix.net/docs/cli/index.html#overview and which turns out to be only applicable to Ubuntu.
Majority of Enterprise S/W runs on RHEL and not Ubuntu - it is difficult to know how would one install bx for RHEL/CentOS as links are buried in your Ubuntu only installer.
Until you find time to support RHEl/CentOS, please update the https://console.bluemix.net/docs/cli/index.html to give the direct download link instead of writing error "'apt-get' is not found.  Thats the only linux installer I know, sorry."

You are correct, the installation script currently only supports Ubuntu.  We've had several requests recently to support RHEL & CentOS, so we're considering those options.   We're also actively working to update content in the docs to make things clearer.
In the meantime, you can manually install the core ibmcloud CLI using the steps listed here:
https://console.bluemix.net/docs/cli/reference/bluemix_cli/get_started.html#getting-started
Then you can install the dev plugin and prerequisites following these instructions:  https://console.bluemix.net/docs/cli/ts_createapps.html#appendix

Here is my steps on install bluemix api on a blue cloud server
`yum install  kubernetes-client etcd git curl
Install helm binary file, the folder linux-amd64 can be removed manually
curl https://storage.googleapis.com/kubernetes-helm/helm-v2.11.0-linux-amd64.tar.gz | tar -zxvf - && mv linux-amd64/helm /usr/local/bin
Install bluemix-cli, the URL cannot be connected on blue cloud server
curl https://plugins.stage1.ng.bluemix.net/download/bluemix-cli/latest/linux64 -o /tmp/bluemix-cli.tgz
tar zxvf /tmp/bluemix-cli.tgz
chmod 755 /tmp/Bluemix_CLI/install
/tmp/Bluemix_CLI/install`


Create burg-halle.txt

Institution/School Name
Burg Giebichenstein Kunsthochschule Halle
Instiution/School Website
http://www.burg-halle.de/
Email Users
burg-halle.de

[x] Students
[x] Academic/Teaching Staff
[ ] Other/Support Staff

Education Levels

[x] Post-graduate research
[x] Graduate School
[ ] College (University) or Undergraduate School or equivalent
[ ] Community College or equivalent
[ ] High School or equivalent
[ ] Elementary School or equivalent

One English information available here:
http://www.burg-halle.de/en/university/studying/information-for-course-applicants/


Add organizational logins

We'll want to add organization associations at some point. What that'll look like is anyone's guess, but I'm putting this here for record keeping.

Putting this issue in the icebox (after consultation w/@cguess of course) because until we better understand the organizational use case, we don't want to bother.

We've decided not to pursue orgs at all; closing this and will be removing organization code in #302.


[Bug] MariaDB fails to restart after the whole cluster rebooted

Describe the bug
I get the folloing error after all nodes where MariaDB insntaces exist has got rebooted
mariadb 2023-10-27  9:13:02 30477 [ERROR] Master 'mariadb-operator': Error reading packet from server: Lost connection to server during
query (server_errno=2013)
mariadb 2023-10-27  9:13:02 30477 [Note] Master 'mariadb-operator': Slave I/O thread exiting, read up to log 'mariadb-bin.000001', posit
ion 912848; GTID position 0-12-61, master mariadb-2.mariadb-internal.seichi-minecraft.svc.cluster.local:3306
mariadb 2023-10-27  9:13:02 30484 [Note] Master 'mariadb-operator': Slave I/O thread: Start semi-sync replication to master 'repl@mariad
b-2.mariadb-internal.seichi-minecraft.svc.cluster.local:3306' in log 'mariadb-bin.000001' at position 1680
mariadb 2023-10-27  9:13:02 30485 [Note] Master 'mariadb-operator': Slave SQL thread initialized, starting replication in log 'mariadb-b
in.000001' at position 1680, relay log<EMAIL_ADDRESS>position: 4; GTID position ''
mariadb 2023-10-27  9:13:02 30484 [Note] Master 'mariadb-operator': Slave I/O thread: connected to master<EMAIL_ADDRESS>al.seichi-minecraft.svc.cluster.local:3306',replication starts at GTID position ''
mariadb 2023-10-27  9:13:03 30485 [ERROR] Master 'mariadb-operator': Slave SQL: Could not execute Write_rows_v1 event on table seichi-ti
med-stats-conifers.break_count_diff_point; Duplicate entry '2099' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_K
EY; the event's master log mariadb-bin.000001, end_log_pos 2067, Gtid 0-12-10, Internal MariaDB error code: 1062
mariadb 2023-10-27  9:13:03 30485 [Warning] Master 'mariadb-operator': Slave: Duplicate entry '2099' for key 'PRIMARY' Error_code: 1062
mariadb 2023-10-27  9:13:03 30485 [ERROR] Master 'mariadb-operator': Error running query, slave SQL thread aborted. Fix the problem, and
 restart the slave SQL thread with "SLAVE START". We stopped at log 'mariadb-bin.000001' position 1680; GTID position '0-12-9'
mariadb 2023-10-27  9:13:03 30485 [Note] Master 'mariadb-operator': Slave SQL thread exiting, replication stopped in log 'mariadb-bin.00
0001' at position 1680; GTID position '0-12-9', master: mariadb-2.mariadb-internal.seichi-minecraft.svc.cluster.local:3306

apiVersion: mariadb.mmontes.io/v1alpha1
kind: MariaDB
metadata:
  namespace: seichi-minecraft
  name: mariadb
spec:
  rootPasswordSecretKeyRef:
    name: mariadb
    key: root-password

  database: seichiassist
  username: mcserver
  passwordSecretKeyRef:
    name: mariadb
    key: mcserver-password

  image:
    repository: mariadb
    tag: "10.11.4"
    pullPolicy: IfNotPresent

  port: 3306

  replicas: 3

  replication:
    enabled: true
    primary:
      podIndex: 0
      automaticFailover: true
    replica:
      waitPoint: AfterSync
      connectionTimeout: 10s
      connectionRetries: 10
      syncTimeout: 10s

  primaryService:
    type: ClusterIP
  primaryConnection:
    secretName: primary-mariadb-conn
    secretTemplate:
      key: dsn

  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - topologyKey: "kubernetes.io/hostname"

  tolerations:
    - key: "mariadb.mmontes.io/ha"
      operator: "Exists"
      effect: "NoSchedule"

  podDisruptionBudget:
    minAvailable: "66%"

  myCnf: |
    [mysqld]
    bind-address=<IP_ADDRESS>
    default_storage_engine=InnoDB
    binlog_format=row
    innodb_autoinc_lock_mode=2
    max_allowed_packet=256M

  volumeClaimTemplate:
    resources:
      requests:
        storage: 100Gi
    storageClassName: synology-iscsi-storage
    accessModes:
      - ReadWriteOnce

  livenessProbe:
    exec:
      command:
        - bash
        - -c
        - mysql -u root -p"${MARIADB_ROOT_PASSWORD}" -e "SELECT 1;"
    initialDelaySeconds: 20
    periodSeconds: 10
    timeoutSeconds: 5

  readinessProbe:
    exec:
      command:
        - bash
        - -c
        - mysql -u root -p"${MARIADB_ROOT_PASSWORD}" -e "SELECT 1;"
    initialDelaySeconds: 20
    periodSeconds: 10
    timeoutSeconds: 5

  metrics:
    exporter:
      image:
        repository: prom/mysqld-exporter
        tag: "v0.14.0"
        pullPolicy: IfNotPresent
      resources:
        requests:
          cpu: 300m
          memory: 512Mi
        limits:
          cpu: 300m
          memory: 512Mi
    serviceMonitor:
      prometheusRelease: prometheus
      interval: 10s
      scrapeTimeout: 10s

Is the bug related to Galera?

[x] I have checked the Galera documentation
[x] I have checked the Galera troubleshooting guide

Expected behaviour

Steps to reproduce the bug

Additional context

Environment details:

Kubernetes version: 1.27.6
mariadb-operator version: 0.20.0
Install method: helm
Install flavour: minimal

Thanks for the detailed information!
This most likely means that the replicas are missing some log operations from the primary and therefore are unable to unable to re-apply fome further operations.
We are looking into automatically fix this situation, adding support physical backups the first thing. The idea is that the operator would detect when the replica Pods are unable to fully replicate and perform the following steps automatically to bring it back:

https://github.com/mariadb-operator/mariadb-operator/issues/141#issuecomment-1804760231

The same applies to adding new replicas to the cluster, we would need to setup the new replica based on a physical backup from the primary, as some log operations may have been expired in the primary i.e. by configuring expire_logs_days. See:

https://mariadb.com/kb/en/using-and-maintaining-the-binary-log/

Related to:

https://github.com/mariadb-operator/mariadb-operator/issues/141

Hey there!
With the release of v0.0.25, primary failover done by the operator has been superseeded by MaxScale: a sophisticated database proxy, router, and load balancer designed specifically for and by MariaDB. I provides a more resilient primary failover based on MariaDB internals, I encourage you to take a look at the docs:
https://github.com/mariadb-operator/mariadb-operator/blob/main/docs/MAXSCALE.md

Hello,
I have the same problems,
I am testing the mariadb operator (one master and 2 replicas) with Proxysql in front.
I am familiar with Proxysql and do not wish to move away from it and switch to MaxScale.
 I kill the same pod again and still the result is the same.
error reconnecting to master 'repl@mariadb-0.mariadb-internal.mariadb.svc.cluster.local:3306' - retry-time: 10  maximum-retries: 100000  message: Can't connect to server on 'mariadb-0.mariadb-internal.mariadb.svc.cluster.local' (111 "Connection refused")

Thank you in advance for your help

This is still in a row


fix: set default policy execution mode to v4-emulation-engine

see https://gravitee.atlassian.net/browse/APIM-2498
Not setting execution_mode to v4-emulation-engine in our model makes execution policy behaviour inconsistent between an API created from the operator and the same API created from the UI.
Tested on 3.20.x and setting this field with this value falls back to v3 there. So that should not introduce any backward compatibility issue.
Waiting confirmation from ENR before making this ready to review.

Waiting confirmation from ENR before making this ready to review.

A fix is coming on APIM as well


VMs error after upgrading to v0.30.0

Is this a BUG REPORT or FEATURE REQUEST?:
/kind bug
What happened:
We're currently running v.0.27.0 on a GCE VMs in europe-west1, using Haskell+ CPUs with nested kvm enabled. We've been running this since about v0.17 without many problems. After upgrading to v0.30.0, VMs no longer want to start and error with this error:
{"component":"virt-launcher","kind":"","level":"error","msg":"Starting the VirtualMachineInstance failed.","name":"app2","namespace":"production","pos":"manager.go:1236","reason":"virError(Code=1, Domain=10, Message='internal error: qemu unexpectedly closed the monitor: 2020-06-16T21:00:57.287355Z qemu-kvm: warning: host doesn't support requested feature: MSR(48BH).vmx-apicv-vid [bit 9]\n2020-06-16T21:00:57.287377Z qemu-kvm: warning: host doesn't support requested feature : MSR(48BH).vmx-vmfunc [bit 13]\n2020-06-16T21:00:57.287403Z qemu-kvm: warning: host doesn't support requested feature: MSR(48DH).vmx-posted-intr [bit 7]\n2020-06-16T21:00:57.287438Z qemu-kvm: warning: host doesn't support requested feature: MSR(48CH).vmx-ept-execonly [bit 0]\n2020-06-16T21:00:57.287476Z qemu-kvm: warning: host doesn't support requested feature: MSR(48CH).vmx-eptad [bit 21]\n2020-06-16T21:00:57.287505Z qemu-kvm: warning: host doesn't support requested feature: MSR(491H).vmx-eptp-switching [bit 0]\n2020-06-16T21:00:57.288240Z qemu-kvm: error: failed to set MSR 0x48b to 0x59ff00000000\nqemu-kvm: /builddir/build/BUILD/qemu-4.2.0/target/i386/kvm.c:2684: kvm_buf_set_msrs: Assertion `ret == cpu-\u003ekvm_msr_buf-\u003enmsrs' failed.')","timestamp":"2020-06-16T21:00:57.378709Z","uid":"2c4bd986-9703-4b80-a81f-4e12f2149e82"}
{"component":"virt-launcher","kind":"","level":"error","msg":"Failed to sync vmi","name":"app2","namespace":"production","pos":"server.go:161","reason":"virError(Code=1, Domain=10, Message='internal error: qemu unexpectedly closed the monitor: 2020-06-16T21:00:57.287355Z qemu-kvm: warning: host doesn't support requested feature: MSR(48BH).vmx-apicv-vid [bit 9]\n2020-06-16T21:00:57.287377Z qemu-kvm: warning: host doesn't support requested feature: MSR(48BH).vmx-vmfunc [bit 13]\n2020-06-16T21:00:57.287403Z qemu-kvm: warning: host doesn't support requested feature: MSR(48DH).vmx-posted-intr [bit 7]\n2020-06-16T21:00:57.287438Z qemu-kvm: warning: host doesn't support requested feature: MSR(48CH).vmx-ept-execonly [bit 0]\n2020-06-16T21:00:57.287476Z qemu-kvm: warning: host doesn't support requested feature: MSR(48CH).vmx-eptad [bit 21]\n2020-06-16T21:00:57.287505Z qemu-kvm: warning: host doesn't support requested feature: MSR(491H).vmx-eptp-switching [bit 0]\n2020-06-16T21:00:57.288240Z qemu-kvm: error: failed to set MSR 0x48b to 0x59ff00000000\nqemu-kvm: /builddir/build/BUILD/qemu-4.2.0/target/i386/kvm.c:2684: kvm_buf_set_msrs: Assertion `ret == cpu-\u003ekvm_msr_buf-\u003enmsrs' failed.')","timestamp":"2020-06-16T21:00:57.378821Z","uid":"2c4bd986-9703-4b80-a81f-4e12f2149e82"}

We're unsure how to fix this and think this is a bug.
What you expected to happen:
We expected that a restart of virt-launcher would just restart the VM, like normal.
How to reproduce it (as minimally and precisely as possible):
That's the hard part, we're not exactly sure what could be the problem here. My guess is that there's something in the cpu features, but we do not set anything in particular. We're using the machine type q35, but that always used before without any issue.
Anything else we need to know?:
Reverting to v0.27.0 (which was quite a hassle by itself) solved the issue and VMs could start again like normal.
Environment:

KubeVirt version (use virtctl version): v0.30.0
Kubernetes version (use kubectl version): v1.15.10
VM or VMI specifications:

apiVersion: kubevirt.io/v1alpha3
kind: VirtualMachine
metadata:
  annotations:
    kubevirt.io/latest-observed-api-version: v1alpha3
    kubevirt.io/storage-observed-api-version: v1alpha3
  labels:
    kubevirt.io/os: linux
    tier: production
    vm-name: app2.domain
  name: app2
  namespace: production
spec:
  running: true
  template:
    metadata:
      annotations:
        cluster-autoscaler.kubernetes.io/safe-to-evict: "false"
      creationTimestamp: null
      labels:
        tier: production
        vm-name: app2.domain
        vm-type: app-server
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: vm-name
                operator: In
                values:
                - app1.domain
            topologyKey: kubernetes.io/hostname
            topologyKey: kubernetes.io/hostname
      domain:
        devices:
          disks:
          - disk: {}
            name: disk0
          interfaces:
          - bridge: {}
            name: default
        machine:
          type: q35
        resources:
          limits:
            cpu: "6"
            memory: 24Gi
          requests:
            cpu: "4"
            memory: 24Gi
      networks:
      - name: default
        pod: {}
      readinessProbe:
        httpGet:
          path: /status/loadbalancer/
          port: 80
        initialDelaySeconds: 120
        periodSeconds: 20
        timeoutSeconds: 30
      volumes:
      - name: disk0
        persistentVolumeClaim:
          claimName: app2

Cloud provider or hardware configuration:

Google Cloud using Haskell+ CPUs with nested kvm support in the images set.

OS (e.g. from /etc/os-release):

Debian GNU/Linux 10 (buster) using the 5.4.0-0.bpo.2-cloud-amd64 kernel from backports.

Kernel (e.g. uname -a):

Linux hostname 5.4.0-0.bpo.2-cloud-amd64 #1 SMP Debian 5.4.8-1~bpo10+1 (2020-01-07) x86_64 GNU/Linux

Install tools:

Kops, mostly.

Others:

*Haswell, not Haskell.

Adding a CPU model like Haswell--noTSX-IBRS solves the issue. I wonder why qemu does not detect the correct CPU features, though, as it has the cpu set to host-model and does some additional checking of features as well.

@timstoop thanks for reporting this.
@ksimon1 @andreabolognani does this sound familiar to anyone? I think there was somewhere a report about a kernel bug or something in combination with gce, but I can't find any details.

@codificat I think your labs are testing on gcp too (https://kubevirt.io/pages/cloud), did you experience that issue?

/cc @vladikr

@rmohr I asked to someone who understands CPU models more than I do, and they said this seems to be the same issue as https://bugzilla.redhat.com/show_bug.cgi?id=1822682. That bug was fixed in qemu-kvm-4.2.0-19 but our COPR contains qemu-kvm-4.2.0-15, so that explains the failure.
I'm not sure what your policy to handle this kind of situation is. I think it would make sense to ask @crobinso to cook up new builds so that what's in the COPR matches what actually got released with RHEL AV 8.2, bugfixes and all, and then maybe release a v0.30.1 which uses the new container?
For the next release we should switch to Fedora builds from the new dedicated COPRS, which are generated automatically so Cole's manual intervention will not be necessary going forward.

I am not sure if we have to backport it. We can also just release it with 0.31. I am all for a backport by @crobinso it is still easier this time. Otherwise I would also be happy if we switch to to the right new COPR. They look great btw. :+1:

Sounds great. If there's any (easy) way for me to test this, let me know, happy to help and provide feedback.

I guess it depends on how frequently you release and how long you support releases once they're out... I'm somewhat familiar with the downstream story, but not with the upstream one :)
Could we switch to the new COPRs right away? What versions of QEMU and libvirt would we use? 4.2.0-7 and 6.0.0-17, which is the closest to what we have right now, or do we want to adopt the newer 5.0.0-2 and 6.4.0-1?

I pushed qemu-kvm-4.2.0-27.el8 to for-kubevirt-8.2 now which contains that fix

Thanks @crobinso! I'll get to work updating the container image right away :)

@andreabolognani @crobinso thanks for pushing the changes.
@timstoop the PR did not make it into the 0.31 release candidate, but our periodic developer build job will release the change in roughly 9 hours. You should see the build here then. Once it ran you can install kubevirt with
$ LATEST=$(curl -L https://storage.googleapis.com/kubevirt-prow/devel/nightly/release/kubevirt/kubevirt/latest)
$ kubectl apply -f https://storage.googleapis.com/kubevirt-prow/devel/nightly/release/kubevirt/kubevirt/$(LATEST)/kubevirt-operator.yaml
$ kubectl apply -f https://storage.googleapis.com/kubevirt-prow/devel/nightly/release/kubevirt/kubevirt/${LATEST}/kubevirt-cr.yaml

Just tested it and this solved my issue indeed! Thank you very much for the swift response, looking forward to the next release. Will this be in a patch release or in 0.31+1?

Sorry for the late reply. Let me cherry-pick it to the 0.31. We can include it in 0.31.1 and release that when the PR is merged.