More guidance on authservice match criteria

I am using the Match criteria so that I can use a single authservice Deployment to authenticate multiple services and hopefully (am yet to test) share state so that you only have to log in once to access multiple services.
I tried using the host header, which I couldn't get to work, but based on this comment I added a header to my authenticated service using a VirtualService
      headers:
        request:
          add:
            x-authservice-match: kiali

which I could then match on in my authservice config
         "match": {
            "header": "x-authservice-match",
            "equality": "kiali"
          },

This feels like a hack - is this expected usage or are there other headers that would be more appropriate?
Additionally the match only works for me if it was all lowercase, for example using  X-Authservice-Match for both the VirtualService and the config fails to match, although my understanding of HTTP headers is that they should be case insensitive.
I am using the latest authservice image: v0.3.1

Hi, I've been looking around for few hours trying to find out how to use host header, and try something like your workaround in Contour but no avail, and turns out it's not host, but :authority!
From this Medium post, section "Send Custom Headers to ext_authz server", it showed us what the headers Envoy sends looks like. Actually I was searching about "Context Extensions" in desperate attempt when I encountered this post.
I don't see :authority documented anywhere, without insight like this it is impossible to make things work. /headbang /facepalm

Envoy's internal HTTP representation is based on HTTP/2 which uses :authority as the Host header: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/route/v3/route_components.proto#envoy-v3-api-field-config-route-v3-routeaction-cluster-header
Additionally, in the Bookinfo example at least, Envoy is configured to call out over HTTP/2: https://github.com/istio-ecosystem/authservice/blob/master/bookinfo-example/config/productpage-external-authz-envoyfilter-sidecar.yaml#L50
In the end I achieved what I was trying to achieve with authservice using oauth2-proxy, which I have written up here: https://www.jetstack.io/blog/istio-oidc/

To answer my own issue, HTTP/2 mandates that header field names are lowercase (compared to HTTP/1.x headers field names which are just case-insensitive). From https://tools.ietf.org/html/rfc7540:

Just as in HTTP/1.x, header field names are strings of ASCII
characters that are compared in a case-insensitive fashion.  However,
header field names MUST be converted to lowercase prior to their
encoding in HTTP/2.  A request or response containing uppercase
header field names MUST be treated as malformed

Also relevant: https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_conn_man/header_casing

I know this is old, but I'm looking to do what you stated in the issue statement. I've looked over your guide before and see that you went with oauth2-proxy, instead. Does that mean there wasn't a less "hacky" way discovered for authservice?


Duplicated code src/crypto/tls and src/tls

Started to use this library in my hobby project, added some fixes to support ClientHello customization and then found that there are at least two folders with duplicated classes.
For example:

https://github.com/bcgit/bc-csharp/blob/master/crypto/src/crypto/tls/AbstractTlsClient.cs
https://github.com/bcgit/bc-csharp/blob/master/crypto/src/tls/AbstractTlsClient.cs

I was worked with src/crypto/tls/AbstractTlsClient.cs, but now it looks that this is a very outdated version and I'm confused.
Could you please explain what is the idea to keep two duplicated folders within the same project?

Org.BouncyCastle.Tls is a new replacement API, recently ported from bc-java, with support for TLS 1.3 amongst other features and improvements. _.Crypto.Tls will be deprecated in a future release once people have had a chance to move to _.Tls and work out any kinks (which won't happen until it's at least appeared in a release). So the duplication is temporary while users migrate.

okay. Is it worth to pull request with some minor fixes for old crypto/tls version?

New _.Tls already has working 1.3, yes (have to specify it in your TlsClient subclass though). I'm happy to apply useful PR to old code (and check if it is relevant to new code).

just tried a new code from src/tls folder. At first attempt it works with a simple request with default ClientHello config, but when I replaced default ClientHello config with a little customized, it fails to verify finished message from www.google.com.
Server negotiate cipher suite 0xCCA8 TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256. Old code works ok with exactly the same ClientHello config.
Here is log file (all values are hex):
send client_hello
       random:       [32 bytes]
       sessionId:
       cipherSuites: cca8,cc13,c02f,c014,c013,009c,0035,002f,000a
         renegotiation_info
         server_name: 0:"www.google.com"
         extended_master_secret
         session_ticket
         signature_algorithms: 0601,0603,0501,0503,0401,0403,0301,0303,0201,0203
         status_request
         next_protocol_negotiation
         signed_certificate_timestamp
         application_layer_protocol_negotiation: "h2","spdy/3.1","http/1.1"
         ec_point_formats: 00
         supported_groups: 0017,0018
         padding
       fingerprint:  771,52392-52243-49199-49172-49171-156-53-47-10,65281-0-23-35-13-5-13172-18-16-11-10-21,23-24,0
       ja3_hash:     3d2a747f5bd0fe4b756a0b19d87e3daf
recv server_hello
       random:       [32 bytes]
       sessionId:
       cipherSuite:  cca8 TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
         extended_master_secret
         renegotiation_info
         ec_point_formats: 00
         session_ticket
         application_layer_protocol_negotiation: "h2"
TLS client negotiated TLS 1.2
recv certificate
recv server_key_exchange
recv server_hello_done
send client_key_exchange
send change_cipher_spec
send finished
recv session_ticket
recv change_cipher_spec
recv finished
send alert: fatal, decrypt_error
TlsFatalAlert: decrypt_error(51)

Tested with Wireshark, ClientHello messages are identical for both cases (byte to byte comparison, the difference is just ClientRandom). For both cases the server https://www.google.com selects cipherSuite cca8 TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, but a new code fails at finished message verification.
Any idea why it may happens?

Could you perhaps summarize the changes you're making to the default ClientHello? I see some new extensions, of which the server appears to accept session_ticket and then actually sends a session_ticket handhsake message.
I note in particular that HandshakeType.new_session_ticket is explicitly excluded from the handshake transcript in current code (see TlsProtocol.ProcessHandshakeQueue). Probably that should NOT be excluded from the transcript pre-TLS 1.3, so I would give that a try.

Here is my GetClientExtension method for my TlsClient customization:
abstract class CustomTlsClient : DefaultTlsClient
{
        public override IDictionary GetClientExtensions()
        {
            IDictionary clientExtensions = TlsExtensionsUtilities.EnsureExtensionsInitialised(base.GetClientExtensions());
            {
                /*
                 * NOTE: If you are copying test code, do not blindly set these extensions in your own client.
                 */
                TlsExtensionsUtilities.AddMaxFragmentLengthExtension(clientExtensions, MaxFragmentLength.pow2_9);
                TlsExtensionsUtilities.AddPaddingExtension(clientExtensions, m_context.Crypto.SecureRandom.Next(16));
                TlsExtensionsUtilities.AddTruncatedHmacExtension(clientExtensions);

                TlsExtensionsUtilities.AddRecordSizeLimitExtension(clientExtensions, 16385);
                TLS.TlsHelper.Add_renegotiation_info(clientExtensions);
                TlsExtensionsUtilities.AddPaddingExtension(clientExtensions, 0);
            }
            var sequence = GetClientExtensionSequence();
            var remove = clientExtensions.Keys
                .Cast<int>()
                .Except(sequence)
                .ToArray();
            foreach (var key in remove)
            {
                clientExtensions.Remove(key);
            }
            var missing = sequence
                .Except(clientExtensions.Keys.Cast<int>())
                .ToArray();
#if true
            foreach (var m in missing)
            {
                clientExtensions[m] = new byte[0];
            }
#else
            if (missing.Length > 0)
                throw new NotImplementedException("missing extensions: "+string.Join(", ", missing.Select(arg=>arg.ToString()).ToArray()));
#endif
            clientExtensions = TLS.TlsHelper.MakeKeyOrderDictionary<int, byte[]>(clientExtensions, pair => GetExtensionOrder(pair.Key, sequence));
            return clientExtensions;
        }

        private int GetExtensionOrder(int type, int[] sequence)
        {
            for (var i = 0; i < sequence.Length; i++)
                if (sequence[i] == type)
                    return i;
            return -1;
        }
//...
}

This code gets extension list from base.GetClientExtensions() and remove all unwanted extensions (which is missing in GetClientExtensionSequence() list) and add extensions which are missing but required. Then it sorts extensions order to comply with required extension sequence and returns IDictionary wrapper which allows to keep Keys list order unchanged to keep extension sequence when it will be written to the stream.
GetClientExtensionSequence is abstract and implemented in the child class which returns required extension sequence in the ClientHello:
        public override int[] GetClientExtensionSequence()
        {
            return new int[] {
                ExtensionType.renegotiation_info,       //(65281)
                ExtensionType.server_name,              //(0)
                ExtensionType.extended_master_secret,   //(23)
                ExtensionType.session_ticket,           //(35)
                ExtensionType.signature_algorithms,     //(13)
                ExtensionType.status_request,           //(5)
                ExtensionType.next_protocol_negotiation,//(13172)
                ExtensionType.signed_certificate_timestamp,             //(18)
                ExtensionType.application_layer_protocol_negotiation,   //(16)
                ExtensionType.ec_point_formats,         //(11)
                ExtensionType.supported_groups,         //(10)
                ExtensionType.padding,                  //(21) [align 512]
            };
        }

Also I removed kludge in TlsProtocol.WriteExtensionsData to write extensions with requested sequence (original code write it in two steps with break requested extension sequence):
        internal static void WriteExtensionsData(IDictionary extensions, MemoryStream buf, int bindersSize)
        {
            WriteSelectedExtensions(buf, extensions);
            WritePreSharedKeyExtension(buf, extensions, bindersSize);
        }
        internal static void WriteSelectedExtensions(Stream output, IDictionary extensions)
        {
            foreach (Int32 key in extensions.Keys)
            {
                int extension_type = key;
                byte[] extension_data = (byte[])extensions[key];

                TlsUtilities.CheckUint16(extension_type);
                TlsUtilities.WriteUint16(extension_type, output);
                TlsUtilities.WriteOpaque16(extension_data, output);
            }
        }

Also I moved adding extended_master_secret from TlsClientProtocol.SendClientHello() and from DtlsClientProtocol.GenerateClientHello(ClientHandshakeState state) to AbstractTlsClient.GetClientExtensions(). Because original code breaks requested extension list. With this change it will be processed before my code to filter and order required extension sequence.
These changes affects ClientHello message generation and the result ClientHello message is correct (tested with Wireshark). And should not affect data flow logic which can cause verify data change.
The same changes I made in old src/crypto/tls code and all worked great with exactly the same ClientHello message and the same cipherSuite.
I tested it on the following url: "https://www.google.com"
Here is the log file for old code src/tls:
send client_hello
       random:       [32 bytes]
       sessionId:
       cipherSuites: cca8,cc13,c02f,c014,c013,009c,0035,002f,000a
         renegotiation_info
         server_name: 0:"www.google.com"
         extended_master_secret
         session_ticket
         signature_algorithms: 0601,0603,0501,0503,0401,0403,0301,0303,0201,0203
         status_request
         next_protocol_negotiation
         signed_certificate_timestamp
         application_layer_protocol_negotiation: "h2","spdy/3.1","http/1.1"
         ec_point_formats: 00
         supported_groups: 0017,0018
         padding
       fingerprint:  771,52392-52243-49199-49172-49171-156-53-47-10,65281-0-23-35-13-5-13172-18-16-11-10-21,23-24,0
       ja3_hash:     3d2a747f5bd0fe4b756a0b19d87e3daf
recv server_hello
       random:       [32 bytes]
       sessionId:
       cipherSuite:  cca8 TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
         extended_master_secret
         renegotiation_info
         ec_point_formats: 00
         session_ticket
         application_layer_protocol_negotiation: "h2"
TLS client negotiated TLS 1.2
recv certificate
recv server_key_exchange
recv server_hello_done
send client_key_exchange
send change_cipher_spec
send finished
recv session_ticket
recv change_cipher_spec
recv finished
send application_data
recv application_data
send application_data
send application_data
send application_data
send application_data
recv application_data
recv application_data
:status: 200
recv application_data
recv application_data
...

as you can see all exactly the same, but old code pass verify message with no issue and process application_data.

Find the following block in TlsProtocol.ProcessHandshakeQueue:
    /*
     * These message types aren't included in the transcript.
     */
    case HandshakeType.hello_request:
    case HandshakeType.key_update:
    case HandshakeType.new_session_ticket:
        break;

and replace it with this:
    /*
     * These message types aren't included in the transcript.
     */
    case HandshakeType.hello_request:
    case HandshakeType.key_update:
        break;

    /*
     * Not included in the transcript for TLS 1.3+
     */
    case HandshakeType.new_session_ticket:
    {
        ProtocolVersion negotiatedVersion = Context.ServerVersion;
        if (null != negotiatedVersion && !TlsUtilities.IsTlsV13(negotiatedVersion))
        {
            buf.UpdateHash(m_handshakeHash);
        }

        break;
    }

Let me know the results.

Tried your change, now it fails at TlsProtocol.CompleteHandshake() here:
                        .SetServerExtensions(m_serverExtensions)
                        .Build();

// fails at the following line, because m_tlsSession==null
                    this.m_tlsSession = TlsUtilities.ImportSession(m_tlsSession.SessionID, m_sessionParameters);
                }
                else
                {
                    securityParameters.m_localCertificate = m_sessionParameters.LocalCertificate;

Just tried with TLS 1.3, also works :)
So the last question is where and how to initialize m_tlsSession properly? Am I doing it in the right way?

I will look into what codepath could leave it uninitialized, but it sounds like it's an OK workaround for now.

Could you please help, where I can catch master secrets to log it for Wireshark packet analysis?
For TLS 1.2 I catch it in TlsProtocol.EstablishMasterSecret(...):
                context.SecurityParameters.m_masterSecret = TlsUtilities.CalculateMasterSecret(context,
                    preMasterSecret);

I write it to NSS log as CLIENT_RANDOM and it works great.
For TLS 1.3 I found secret calculation at TlsUtilities.Establish13PhaseSecrets(...), but here is a lot of calculations and I'm not sure where is what I need.
Wireshark expects the following secrets in the log file for TLS 1.3:

CLIENT_HANDSHAKE_TRAFFIC_SECRET handshake traffic secret for the client side (for TLS 1.3)
SERVER_HANDSHAKE_TRAFFIC_SECRET handshake traffic secret for the server side (for TLS 1.3)
CLIENT_TRAFFIC_SECRET_0 application traffic secret for the client side (for TLS 1.3)
SERVER_TRAFFIC_SECRET_0 application traffic secret for the server side (for TLS 1.3)
EXPORTER_SECRET exporter secret (for TLS 1.3)

where I can catch these secrets in the code?
Thanks

The issue with m_tlsSession is caused around line 711 of TlsClientProtocol when the new_session_ticket is received. It clears any existing session, but should be resetting the sessionID and restoring m_tlsSession accordingly. Both this fix and the handshake hash update fix are now pushed to our git repo and should mirror here soon (also fixed in bc-java).


port CI changes to release branches

The changes in the CI scripts in https://github.com/LSSTDESC/CCL/pull/912 likely need to be ported to release branches.

Sorry this is probably silly but I don't really understand what you mean by release branches. The releases are made off ther master branch - what am I missing? @beckermr

There are branches that are named release/xyz. We should port the changes that fixed read the docs and the ci scripts we made late last year to these branches.
Sorry for writing such a vague issue!


iOS version

Hi!
Great plugin!
The Android version works alot better than the iOS version, what SDK is used in the Android is it possible to use the same in the iOS version?
BR Patrick

Android uses android's native speech recognizer ( http://developer.android.com/reference/android/speech/SpeechRecognizer.html ), so no, I can't use the same in the iOS version. If you know a better SDK for iOS speech recognition let me know.

I am new to this area.
Why can't you use Google speech-to-text API.
Here is a project sample for doing in that way.

 (I don't recommend it)
The API only accepts up to ~10-15 seconds of audio.
Generating your own Speech API Key, you can only make 50 requests per day.

So I don't think it's suitable


Update Rest API yaml files

Purpose
Updated following yaml files

admin-api.yaml
publisher-api.yaml
devportal-api.yaml
gateway-api.yaml

Related issues:
Fix https://github.com/wso2/docs-apim/issues/5959
Fix https://github.com/wso2/docs-apim/issues/5954
Fix https://github.com/wso2/docs-apim/issues/5948
Fix https://github.com/wso2/docs-apim/issues/5932
Goals

Describe the solutions that this feature/fix will introduce to resolve the problems described above

Approach

Describe how you are implementing the solutions. Include an animated GIF or screenshot if the change affects the UI (email<EMAIL_ADDRESS>to review all UI text). Include a link to a Markdown file or Google doc if the feature write-up is too long to paste here.

User stories

Summary of user stories addressed by this change>

Release note

Brief description of the new feature or bug fix as it will appear in the release notes

Documentation

Link(s) to product documentation that addresses the changes of this PR. If no doc impact, enter “N/A” plus brief explanation of why there’s no doc impact

Training

Link to the PR for changes to the training content in https://github.com/wso2/WSO2-Training, if applicable

Certification

Type “Sent” when you have provided new/updated certification questions, plus four answers for each question (correct answer highlighted in bold), based on this change. Certification questions/answers should be sent to<EMAIL_ADDRESS>and NOT pasted in this PR. If there is no impact on certification exams, type “N/A” and explain why.

Marketing

Link to drafts of marketing content that will describe and promote this feature, including product page changes, technical articles, blog posts, videos, etc., if applicable

Automation tests

Unit tests

Code coverage information

Integration tests

Details about the test cases and coverage

Security checks

Followed secure coding standards in http://wso2.com/technical-reports/wso2-secure-engineering-guidelines? yes/no
Ran FindSecurityBugs plugin and verified report? yes/no
Confirmed that this PR doesn't commit any keys, passwords, tokens, usernames, or other secrets? yes/no

Samples

Provide high-level details about the samples related to this feature

Related PRs
https://github.com/wso2/docs-apim/pull/5930
https://github.com/wso2/docs-apim/pull/6002
Migrations (if applicable)

Describe migration steps and platforms on which migration has been tested

Test environment

List all JDK versions, operating systems, databases, and browser/versions on which this feature/fix was tested

Learning

Describe the research phase and any blog posts, patterns, libraries, or add-ons you used to solve the problem.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.2 out of 3 committers have signed the CLA.:white_check_mark: ashera96:white_check_mark: dulanjalidilmi:x: Dilmi DulanjaliDilmi Dulanjali seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


fixed copied command executable path in Dockerfile

only executable file should be copied, but currently entire directory of bridge repository is copied.

Codecov Report

Merging #13 (59e6ac3) into main (c47dc82) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #13   +/-   ##
=======================================
  Coverage   55.04%   55.04%
=======================================
  Files          14       14
  Lines         625      625
=======================================
  Hits          344      344
  Misses        262      262
  Partials       19       19

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c47dc82...59e6ac3. Read the comment docs.