Devops: Get a dedicated SSL cert for Prod

From Nick Corcoran @ CITZ reached me in teams
Hi Meghan. I just saw the announcement of the Digital Telework Agreement (Great Job) and see that you're the product owner. I would ask that you obtain a dedicated TLS (SSL) certificate for your production site, and not use the platform wildcard certificate. The wildcard cert is fine for non-prod, but all prod sites need a dedicated certificate. These can be obtained through the following process: https://developer.gov.bc.ca/BC-Government-OpenShift-DevOps-Security-Considerations#tls-certificates
request: get a dedicated TLS (SSL) certificate for production site, and not use the platform wildcard certificate.
@warrenchristian1telus : Can you advise about the urgency of this request? I'm thinking it could wait until next week, but I'd welcome your recommendation.

I have email Igor to assist

@MeghanStothers I would recommend requesting the following sub-domains prior to requesting certificates for the same:

digital-journeys.gov.bc.ca
analytics.digital-journeys.gov.bc.ca
api.digital-journeys.gov.bc.ca
camunda.digital-journeys.gov.bc.ca
files.digital-journeys.gov.bc.ca
forms-flow.digital-journeys.gov.bc.ca

I'm wondering if this may be better off as an Epic, so that we can create sub-issues for requesting the domains, related SSL certificates, and the subsequent implementation & testing.

Check Peter Velinov

Request submitted by email to IT Service desk/Peter Velinov with a request to follow up with Stella as I am away until August 29.

Can someone assist and send off the iStore request to an approver? Thank you. I am not able to attach the email, so I'll send it to Alison and Stella by email.

Peter Velinov and Kunal have connected with Chris Brown at TES to assist in getting these SSL certs.

Peter Velinov is going to confirm with Chris Brown (TES) whether he is going to create the SSL certs for us or if Peter needs to submit an iStore to OCIO to have them created. Whatever avenue we take, once the certs are received, Kunal will install them for us.

Chris Brown (TES) is creating these for us and once completed, Warren and/or Kunal will implement them for us.

@kkapoor-fresh is there any progress on this one? Last I saw Chris was creating the certs for us...

@Abuchana I haven't heard back from Chris on this yet. I have sent a followup email to him right away

@kkapoor-fresh @warrenchristian1telus Hey, do you have any advice ont his ticket? Is this urgent?

@fazil-ey @kkapoor-fresh I don't think this is urgent as we already have SSL - just not the custom cert. I do have the new certificate available when someone has time to implement. Just let me know who/where I need to send it to.

Waiting for @kkapoor-fresh to action this

@kkapoor-fresh is there anyway we can action this anytime soon?

@fazil-ey, @warrenchristian1telus mentioned that this might not be an item that is urgent. Let me know if it is and I can take a look at it next week.

I don't see a problem with that @kkapoor-fresh. I know you're busy with PECSF. I have the certs for this, if you don't have them yet, so just let me know if/when you need them.

We've had some issues with SSL implementation. The current state of the new certificates is as follows:
Webapp - Waiting on new certificate
Webapi - Waiting on new certificate
Camunda - Reverted due to application errors
Forms flow - Reverted due to application errors
Files - Reverted due to application errors
Anlytics - In production
@kkapoor-fresh Will continue to proceed with installations and troubleshooting. Please update this ticket to reflect changes as they become available.

Following up with Chris for the new certs.

@kkapoor-fresh Checking in on this one while warren is off - can we help in any way? Is the Chris you're referring to with the DJ team or with TES?  Thanks for moving this along

@kkapoor-fresh We are launching on March 17, 2023. Let us know if you have any updates on this from Chris

Just got new certs from Chris. I will test them out and see if they work fine.

@Abuchana @warrenchristian1telus I tested the new certs. The cert for the web api seems to be working fine in Prod. Will monitor it through Monday
The cert for web app is still not correct. I have sent a note to Chris to check that one.That one might need to be resent.

Everything should work as expected - to send Igor the latest update on this one

Iman has tested every thing working fine - error free

from Warren: we have one of two tickets. One is outstanding for docs.

@warrenchristian1telus
forms-flow-document URL was shortened on DEV to comply with the SSL certificate length limitation. I tested PDF generation and it worked well.
TEST and PROD are still waiting for that change.

Warren trying to track down right key

We have received renewed SSL certificates for all domains other than documents.
The earliest expiry dates are Fri, 08 Sep 2023 - so we should implement all before then to avoid errors / interruptions.
To resolve the issue with a missing key for documents domain, I have requested a new certificate be created for digital-journeys-docs.apps.silver.devops.gov.bc.ca. I'm hoping this will be available to us prior to Sept. 8th so we can implement all of them at once.

Hi @MeghanStothers @Abuchana @Stella-Archer @iman-jamali-fw - I had a chat with @kkapoor-fresh earlier, and we thought it would be safer to implement this tonight, just in case we have any surprises with the SSL certificates that expire on Friday. It may be nice to have additional time to get external support if something doesn't work as expected.
If we can get a quick approval for this, and hopefully Iman's help, we can schedule for after 4pm this evening. Otherwise, we can proceed tomorrow afternoon as previously discussed.

@warrenchristian1telus that makes sense to me and will go with whatever the devs think is best. I have to go out after work today but would be back home at 5:30 if you need me for anything.

@warrenchristian1telus
I'm good for both this afternoon or tomorrow.

Thanks!  I'll be here to support until about 5:30 or 6, although I will not have any PSA devs to support/test. I can certainly ask Ayush to test after- hours if you let me know what tests we should be performing.

@MeghanStothers
One sample submission from each live form would be great to ensure different components (website, API, Camunda, PDF generation server) are working OK with the new SSL certs. I'll do the initial test and can submit one Telework test submission.

Thanks @iman-jamali-fw. @kkapoor-fresh can we schedule this to start at 4pm? I'm hoping we'll be able to test somewhere between 4:30-5pm.

@warrenchristian1telus Yes, 4 PM looks OK.

@warrenchristian1telus
I'm good with 4 pm too!

@Abuchana All set for 4 today.
@iman-jamali-fw I can help test the Telework (pdf) as well as form access from the website, I'll need other devs to check Camunda, API etc.

I would also like to mention that we not only have to confirm that it still works, but that the certificate dates have successfully updated to 2024.
Private Zenhub Image

@MeghanStothers @Abuchana @iman-jamali-fw @kkapoor-fresh
All SSL certificates have been updated (except documents). Please verify and let us know if any of you spot any issues.

@warrenchristian1telus
Thank you. I'll get to the testing.

Awesome! I submitted a telework agreement at 5:02 today and sent to<EMAIL_ADDRESS>in case she can intercept.
Findings -

I tested in the live prod environment
I was able to begin the digital submission from the Careers & MyHR website
My submission was successful in terms of email messages as expected, as well as the pdf was attached to the email and displays as expected.
I can test with someone else with a gov address to make sure the supervisor's role delivers as expected

I've verified all the domains now provide the new certs which expire in Sept. 2024. You may need to clear your cache to see the changes. Everything is looking good on my end.

I checked all the components and they're working great with the new SSL certs.
Also sent one Telework with myself as employee and manager, received both emails with the PDF attached. This means all components (website, API, Camunda and PDF generation) works OK with the new certs.

Excellent - unless there are further tests, this one is a go!
I have asked @chriscaldwell-psa to keep an eye on the DJ mailbox just in case we get any issues in the morn. Thanks all!

Great news - thanks @iman-jamali-fw!
If nobody else can find any issues, I believe this ticket can be closed.

We'll watch for any further issues. Happy to close this one...and stay closed :)  Thanks all


Adds action to store media using spatie/laravel-medialibrary

Purpose
By default, the media files are stored in the public disk in the media directory.
This adds the possibility to store media using spatie/laravel-medialibrary package.
The user will need to change the media_action key in the config file filament-tiptap-editor.php to SpatieMediaLibraryAction:

Good points. Thank you.
I'll correct the ck-media thing and add alt etc

I fixed the ck-media mistake.
Concerning the alt, title and dimensions there is no need to implement anything using spatie/laravel-medialibrary because those parameters are stored in the text itself as the  attributes eg. like this:
<img src="/storage/11/conversions/29e032e5-06b5-43cf-867c-80f33809801d-large.webp" alt="Alt Text" title="The photo title" width="1280" height="853" loading="lazy">
Is this OK?

Wondering if this needs to be in the plugin itself? Seems like if could just be a Class in the app, since it can still be changed via the config.

Well, you are the boss. But I think not. At lease no more than the "standard" way of storing files. I think adding this functionality do the core of the plugin will be beneficial taking under consideration how popular spatie/laravel-medialibrary is.
The very big downside to the standard way of storing files is that there is no control over the files (eg. converting/optimising  them_ and even worse they are not deleted when the parent record is deleted.
I know it is possible to use your Curator plug-in but I can't do it because I already use spatie's one and changing this would be a nightmare. I believe this can be true for many many others.

Valid points.
How would you handle inserting the converted images back into the editor? i.e, upload creates a 'medium' sized image, where would that option come to insert that size as the image in the editor. And also what if it takes a while to generate that image, you would then need a placeholder to show in the editor, which would require updates to the JS extension internally.
Not totally opposed to including this, I just think there's more to it that would require some further troubleshooting, or maybe even a better implementation of something like a 'Media Provider' class or similar.

You are right - this can be a problem as everybody can name the conversion as they see fit...
The 'large' or 'medium' comes from the model.
For example:
 public function registerMediaConversions(Media $media = null): void
    {
        $this->addMediaConversion('thumb')->fit(Fit::Crop, 50, 50)->optimize()->format('webp');
        $this->addMediaConversion('preview')->fit(Fit::Crop, 450, 300)->optimize()->format('webp');
        $this->addMediaConversion('large')->fit(Fit::Crop, 850, 548)->optimize()->sharpen(10)->format('webp');
    }

public function getFeaturedImageAttribute()
    {
        $file = $this->getMedia('featured_image')->last();

        if ($file) {
            $file->url       = $file->getUrl();
            $file->thumbnail = $file->getUrl('thumb');
            $file->preview   = $file->getUrl('preview');
            $file->large     = $file->getUrl('large');
        }

        return $file;
    }

So using my spatie solution you should be able to somehow decide which conversion (if any) you want to use.
I think this should be set by adding  something like this (so with every model could use different conversion):
TiptapEditor::make('main_text')->useSpatieConversion('large'),
Or add it in setting file to use in all models.
What do you think?

Honestly, I think this is assuming a lot. But I do see the value, despite knowing, at least right now, the best way forward. Like I'm torn on supporting something I don't personally use, but at the same time it's kind of the point of being able to provide your own custom Media class.

I'm going to close this for now. Thank you for your hard work on this. I will consider it more for v4, but I feel that this is still better handled outside the core plug-in.
If you'd like to release it as an add on for Curator, I'd be OK with it. Cheers.


mesh of real dataset

Hi, I find part of the reconstructed mesh is missing. I guess tuning some parameters can be helpful. The following is a screenshot of the Bronze reconstruction (200k iteration). I use the default parameters which are also listed below. May I please ask for any suggestions?

train {
    learning_rate = 5e-4
    learning_rate_alpha = 0.05
    end_iter = 200000

    batch_size = 512
    validate_resolution_level = 4
    warm_up_end = 5000
    anneal_end = 50000
    use_white_bkgd = False

    save_freq = 10000
    val_freq = 1000
    val_mesh_freq = 10000
    report_freq = 100

    igr_weight = 0.1
    mask_weight = 0.0
}

model {
    nerf {
        D = 8,
        d_in = 4,
        d_in_view = 3,
        W = 256,
        multires = 10,
        multires_view = 4,
        output_ch = 4,
        skips=[4],
        use_viewdirs=True
    }

    sdf_network {
        d_out = 257
        d_in = 3
        d_hidden = 256
        n_layers = 8
        skip_in = [4]
        multires = 6
        bias = 0.5
        scale = 1.0
        geometric_init = True
        weight_norm = True
    }

    variance_network {
        init_val = 0.3
    }

    rendering_network {
        d_feature = 256
        mode = idr
        d_in = 9
        d_out = 3
        d_hidden = 256
        n_layers = 4
        weight_norm = True
        multires_view = 4
        squeeze_out = True
    }

    neus_renderer {
        n_samples = 64
        n_importance = 64
        n_outside = 32
        up_sample_steps = 4     # 1 for simple coarse-to-fine sampling
        perturb = 1.0
    }
}

I think you could increase the resolution of the function ‘validate_mesh'
to 512 and manually adopt a bounding box to filter mesh.
On Thu, Jul 25, 2024 at 2:39 PM Yanhao Zhang @.***>
wrote:

Hi, I find part of the reconstructed mesh is missing. I guess tuning some
parameters can be helpful. The following is a screenshot of the Bronze
reconstruction (200k iteration). I use the default parameters which are
also listed below. May I please ask for any suggestions?
image.png (view on web)
https://github.com/user-attachments/assets/48cd7045-2d47-4061-b62e-98c833fb1da6
train {
learning_rate = 5e-4
learning_rate_alpha = 0.05
end_iter = 200000
batch_size = 512
validate_resolution_level = 4
warm_up_end = 5000
anneal_end = 50000
use_white_bkgd = False

save_freq = 10000
val_freq = 1000
val_mesh_freq = 10000
report_freq = 100

igr_weight = 0.1
mask_weight = 0.0

}
model {
nerf {
D = 8,
d_in = 4,
d_in_view = 3,
W = 256,
multires = 10,
multires_view = 4,
output_ch = 4,
skips=[4],
use_viewdirs=True
}
sdf_network {
    d_out = 257
    d_in = 3
    d_hidden = 256
    n_layers = 8
    skip_in = [4]
    multires = 6
    bias = 0.5
    scale = 1.0
    geometric_init = True
    weight_norm = True
}

variance_network {
    init_val = 0.3
}

rendering_network {
    d_feature = 256
    mode = idr
    d_in = 9
    d_out = 3
    d_hidden = 256
    n_layers = 4
    weight_norm = True
    multires_view = 4
    squeeze_out = True
}

neus_renderer {
    n_samples = 64
    n_importance = 64
    n_outside = 32
    up_sample_steps = 4     # 1 for simple coarse-to-fine sampling
    perturb = 1.0
}

}
—
Reply to this email directly, view it on GitHub
https://github.com/JiaxiongQ/NeuS-HSR/issues/8, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJANJRD6GFQOQ4O42RRZOALZOCMTHAVCNFSM6AAAAABLN35N6GVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQZDSMJUGY3TSNA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Thanks a lot for this information.


[WIP] Revisit some PR 553 performance changes

I'm measuring whether some performance changes introduced in PR #553 actually brought performance gains or not.

I'm measuring whether some performance changes introduced in PR #553 actually brought performance gains or not.

If I remember correctly, I think it will make more difference when the upstream calls are updated as well.

Are you able to compare database size and freelist pages after execution with and without this PR ?
toolbox --datadir <whatever> tables
Besides I do have improvements in speed too. (IntelNUC 32GB RAM 2TB NVMe)

Are you able to compare database size and freelist pages after execution with and without this PR ? toolbox --datadir <whatever> tables
Besides I do have improvements in speed too. (IntelNUC 32GB RAM 2TB NVMe)

OK, I'll try to compare database size and freelist pages, but it'll take a few days.
What was your OS, compiler, and the speed improvement you observed?
My test machine in Google Cloud is Ubuntu 21.10 with Intel Xeon CPU @ 2.80GHz, 64GB RAM and 3TB NVMe. Compiler: gnu 11.2.0.


Added mimetypes for json and javascript files

Fixes: https://github.com/Azure/azure-cli/issues/6257
Fixes: https://github.com/Azure/azure-cli/issues/6402
This checklist is used to make sure that common guidelines for a pull request are followed.

[x] The PR has modified HISTORY.rst describing any customer-facing, functional changes. Note that this does not include changes only to help content. (see Modifying change log).

Codecov Report

Merging #6461 into dev will not change coverage.
The diff coverage is n/a.

@@         Coverage Diff         @@
##           dev   #6461   +/-   ##
===================================
  Coverage    0%      0%
===================================
  Files       11      11
  Lines      133     133
  Branches     9       9
===================================
  Misses     133     133

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c2fa315...3f3f6b6. Read the comment docs.


fix #62 - updated layout-worklets to async/await

Not sure how the "relative" demo was supposed to work... otherwise, all is working properly again 🤩

@googlebot I signed it!

Thanks @d4tocchini ! Much appreciated!


Refactor LocalizedError

Love it! I say merge 👍

Closing this PR.  See PR #327 for these changes after merging with the recent project refactoring in the develop branch.


Pass file object rename function

For some particular situations I've needed that work watching the file stream. For instance when the src parameter is an array.

gulp
  .src(['/scripts/main.js', 'styles/main.styl'])
  .rename(function(path, file){
    // work like always but now file stream is here
  });

Best!

:+1:

@retrofox
Could you show us more detailed use case?
I'm afraid that this API might be used outside the use of rename plugin.

Closing this PR since we can use through2 directly in this case.
var gulp = require('gulp');
var through = require('through2');

gulp.src('src/*.js')
  .pipe(through.obj(function(file, enc, cb) {
    // process
  }));

Perhaps there are other reasons that aren't mentioned here, but rejecting this based on the fact that it can be done with through2 seems moot. We could directly use through2 in the .pipe(rename(function(path) {...})) case, too, yet the functionality is there.


Scroll up not propagating to parent when at top of "short" grid

I've created this plunk[1] to show the bug.
Scroll down while keeping your mouse inside the grid, keep your mouse in grid and, now try to scroll up.
This bug doesn't seem to appear in the 3.0.0-rc.19 release.
Is there a solution to this in the latest 3.0.0-rc.20 release?
[1] http://plnkr.co/edit/ke8jCXG7qsOzWRqOsrw1?p=preview

This is working on the tutorial: http://ui-grid.info/docs/#/tutorial/101_intro, I assume it's a difference between rc20 and unstable, and is therefore already fixed in unstable.

Thanks for the quick response @PaulL1. So, I guess I should downgrade to 3.0.0-rc.19 until new version is released.
Thanks for the awesome work :)

I'd be more inclined to upgrade to unstable, there's quite a bit in there.  I can't recall what other bugs you're inheriting by going back to rc19.

Ah! then it'll make sense to use the unstable. What would you say is the best way, to get the unstable version? Right now I'm getting the rc20 from bower.

Easiest to get it from ui-grid.info/release. Check the tutorials for links.
You can also clone the repo and build, but that's a bit fiddlier.
Sent from my iPhone

On 23 Apr 2015, at 5:03 pm, Gaurav Butola<EMAIL_ADDRESS>wrote:
Ah! then it'll make sense to use the unstable. What would you say is the best way, to get the unstable version? Right now I'm getting the rc20 from bower.
—
Reply to this email directly or view it on GitHub.

Great! thanks for your help.

I am implementing the latest version and I am also experiencing this issue. I have several small grids on a page (<= 10 rows) and the user is unable to scroll up while mouse is over the grid. The vertical scrollbar is disabled. I do also notice that this issue can be reproduced on the tutorial that you sited above. Any updates? I have just finished a massive refactor so I am not overly concerned about his issue, but my users might. Thanks so much!

Hi, I am also experiencing this issue. And it doesn't work on http://ui-grid.info/docs/#/tutorial/101_intro .
This problem can be reproduced on a grid that has no scroll bar activated.
I try to debug on the last unstable version (RC21). I add some comment, if it helps you to find what's wrong, I don't want to try to presume what the problem ...
if (event.deltaY !== 0) {
    // On scroll to top, scrollYAmount < 0 (-53 for me). On scroll to bottom, scrollYAmount > 0
    var scrollYAmount = event.deltaY * -1 * event.deltaFactor;

    scrollTop = containerCtrl.viewport[0].scrollTop; // scrollTop === 0
    // Get the scroll percentage
    // -- scrollTop + scrollYAmount < 0 (-53 for a scroll to top)
    // -- rowContainer.getVerticalScrollLength() < 0 (-299 for a grid height of 300px)
    var scrollYPercentage = (scrollTop + scrollYAmount) / rowContainer.getVerticalScrollLength();
    // -- 0 < scrollYPercentage < 1, but the scroll Y percentage should be 0, no ?

    // -- If it's a scroll to bottom, the scrollYPercentage will be < 0 (
    // -- so scrollPercentage will be equal to 0

    // Keep scrollPercentage within the range 0-1.
    if (scrollYPercentage < 0) { scrollYPercentage = 0; }
    else if (scrollYPercentage > 1) { scrollYPercentage = 1; }

    scrollEvent.y = { percentage: scrollYPercentage, pixels: scrollYAmount };
}
// [...]
// Let the parent container scroll if the grid is already at the top/bottom
if (event.deltaY !== 0 && (scrollEvent.atTop(scrollTop) || scrollEvent.atBottom(scrollTop))) {
    // -- It doesn't enter here if it's a scroll to top
} else {
    event.preventDefault();
    // [...]
}

I hope this will help you ...
Oh and thanks for this module !

Running into the same issue using the unstable release from yesterday (v3.0.0-rc.21-c0d9b48).

:+1: For this issue, I can't scroll my page up at all because I have a 100% width table.

The scroll down have some issue too. I make some test :

If the canvas is smaller than the view port, and there's no scroll : my first PR can handle this case.
If the canvas is smaller than the view port, and there's a scroll : this case is a real problem, indeed, the code calculate a scrollYPercentage equal to 0, even is the scroll is at the bottom. (http://plnkr.co/edit/Xoianpl2wWSKuQVtuxOt?p=preview)
If the canvas is bigger than the view port : no problem

I make a new PR (#3666) to ensure all case


Uncaught Error: Cannot call function 'getPosition' on missing remote object 45

[Enter steps to reproduce:]

...
...

Atom: 1.41.0 x64
Electron: 4.2.7
OS: Mac OS X 10.15.1
Thrown From: Atom Core
Stack Trace
Uncaught Error: Cannot call function 'getPosition' on missing remote object 45
At /Applications/Atom.app/Contents/Resources/electron.asar/renderer/api/remote.js:222

Error: Cannot call function 'getPosition' on missing remote object 45
    at throwRPCError (/Applications/Atom.app/Contents/Resources/electron.asar/browser/rpc-server.js:148:17)
    at /Applications/Atom.app/Contents/Resources/electron.asar/browser/rpc-server.js:406:5
    at EventEmitter.ipcMain.on.args (/Applications/Atom.app/Contents/Resources/electron.asar/browser/rpc-server.js:273:21)
    at EventEmitter.emit (events.js:182:13)
    at WebContents.<anonymous> (/Applications/Atom.app/Contents/Resources/electron.asar/browser/api/web-contents.js:368:21)
    at WebContents.emit (events.js:182:13)

Commands

Non-Core Packages
atom-clock 0.1.17
hey-pane 1.2.0
Hydrogen 2.13.1
scroll-through-time 0.3.1
teletype 0.13.4
zentabs 0.8.9

Thanks for contributing!
We noticed that this looks like a duplicate of https://github.com/atom/atom/issues/16630 so you can subscribe there for updates if you'd like.
If the error is reproducible for you, please let us know the steps to reproduce and if you can reproduce in safe mode.

Because we treat our issues list as the Atom team's backlog, we close duplicates to focus our work and not have to touch the same chunk of code for the same reason multiple times. This is also why we may mark something as duplicate that isn't an exact duplicate but is closely related.
For information on how to use GitHub's search feature to find out if something is a duplicate before filing, see the How Can I Contribute? section of the Atom CONTRIBUTING guide.


Routr docker not booting up

Hi,
I'm running routr with docker using following commandline:
docker run -it \
    -p 4567:4567 \
    -p 5060:5060 \
    -p 5060:5060/udp \
    -e EXTERN_ADDR=`curl ifconfig.co` \
    -e DATA_SOURCE_PROVIDER=files_data_provider \
    -v $(pwd)/config:/opt/routr/config \
    fonoster/routr

with config mounted locally with default content.
The routr exits with error:
[INFO  tini (1)] Main child exited normally (with status '0')
[INFO  tini (1)] Spawned child process './routr' with pid '9'
[ERROR] com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
 at [Source: (StringReader); line: 19, column: 20]
[INFO  tini (1)] Main child exited normally (with status '0')

What am I missing?

Got it - I don't have gateways.yml configured. But this is what I want - I want only alone routr instance to make calls using sip within it. Is it possible?

Got it - I don't have gateways.yml configured. But this is what I want - I want only alone routr instance to make calls using sip within it. Is it possible?

Hey! @nyxnyx
Yes, you can use Routr without using gateways. That Is a legit use case. We do have a pending issue that causes Routr to fail to start if there is a missing config file. I'm closing this, but feel free to continue the conversation at #92

Hey! @nyxnyx
Yes, you can use Routr without using gateways. That Is a legit use case. We do have a pending issue that causes Routr to fail to start if there is a missing config file. I'm closing this, but feel free to continue the conversation at #92


Add overseerr support

Adding overseerr support by following the same model used for other containers.  This is my first github contribution so thanks in advance for your patients and understanding.

I just wanted to give you a little background on why i added main.py logic to ask for overseerr if radarr or sonarr is selected and in the media center area if plex is selected.  The reason is I'm aware of a few that use ezarr just for the Arrs but not for the plex server.  I suppose with that logic that overseerr question could be all on its own and not an IF clause for radar and sonarr.  I'm happy to make that change if you think its best.

Resolves #24

sorry closing due to dupe.


impl(common): a lock class with ABSL_GUARD* support

This allows the google-cloud-cpp library to use std::mutex, scoped locking and the safety from checked thread annotations at the same time.

This change is

I think I will take a pass on this.  Not see the value and I do not need to learn another non-standard-C++ version.

Slightly more canonical commit / description.

@dpcollins-google offline, you said that this PR was the result of an error you got from an earlier PR. Can you link to that error so we can understand what this is fixing?


Fix CLI auth

This fixed the CLI auth on my side #3

Handled by #5 Note, in that PR the logic stays api/auth/cli.rb Thanks for the PR though


How to change visible or hidden edge color after contructing OutlineEffect

I simply cannot figure out how to change the hidden or visible edge colors of the outline effect after it has already been constructed. Can't seem to find docs anywhere showing me how to do this. Any way you can let me know how or if its even possible?
this.outlineEffect = new OutlineEffect(this.scene, this.selectedCamera, {edgeStrength: 2.0, visibleEdgeColor: '#FFFFFF', hiddenEdgeColor: '#FF0000', blur: true, blendFunction: THREE.NormalBlending});

// Want to be able to change the color later if certain requirements are met
this.outlineEffect.hiddenEdgeColor = '#FFFFFF'

Hi,
the OutlineEffect extends Effect which defines uniforms as a Map<String, Uniform>. You can adjust the edge color like this:
const uHiddenEdgeColor = outlineEffect.uniforms.get("hiddenEdgeColor");
uHiddenEdgeColor.value.setHex(0xffffff);

// Might need to convert to linear depending on where the color is coming from.
// uHiddenEdgeColor.value.convertSRGBToLinear();

Please note that the OutlineEffect expects hexadecimal values for the edge colors. Strings like "#FFFFFF" won't work.

Hey, thanks so much this worked fo me. Really Appreciate it.


VTN create simple DR event

Hi Stan,
From the documentation for the VTN server (code below) examples I am experimenting with modifying for open ADR LOAD_CONTROL, specifically LoadControlCapacity
# Add a prepared event for a VEN that will be picked up when it polls for new messages.
server.add_event(ven_id='ven_id_123',
                 signal_name='LOAD_CONTROL',
                 signal_type='LoadControlCapacity',
                 intervals=[{'dtstart': datetime(2021, 2, 1, 13, 0, 0, tzinfo=timezone.utc),
                             'duration': timedelta(minutes=60),
                             'signal_payload': 1.0}],
                 callback=event_response_callback)

From open ADR documentation for profile B, its this:

These instructions are used to set the load control to values that are relative to the load controller and its output capacity. This does not require the VTN or the VEN knowing precisely what the load consumption level is, but are expressed in ways that the VTN can know that the signal values will either increase or decrease the load consumption regardless of the specific type of device that is performing the load control. These can be used for some aspects of direct load control by mapping these general instructions to specific load control commands in the VEN without the VTN needing to know precisely what device may be consuming the signal.

If the VTN code snip it above seems Ok, how would I see this signal_payload on the VEN side which can then be linked up to some more synchronous BACnet write commands to shut equipment down? I'm still experimenting with the same code from issue #51 which has some ongoing discussions on preferred way to handle BACnet read/writes.
import asyncio
from datetime import timedelta
from openleadr import OpenADRClient, enable_default_logging
import BAC0
import concurrent.futures
import numpy
from functools import partial

enable_default_logging()

#apprently runs on its own thread
bacnet = BAC0.lite()

def blocking_io():
    #read temp sensor from BACnet device
    try:
        sensor_value = bacnet.read('12345:2 analogInput 2 presentValue')
        return sensor_value
    except:
        return numpy.nan

async def collect_report_value():
    # 1. Run in the default loop's executor:
    result = await loop.run_in_executor(None, blocking_io)
    return result

async def handle_event(event):
    # This callback receives an Event dict.
    # You should include code here that sends control signals to your resources.
    return 'optIn'

# Create the client object
client = OpenADRClient(ven_name='ven123',
                       vtn_url='http://localhost:8080/OpenADR2/Simple/2.0b')

# Add the report capability to the client
client.add_report(callback=collect_report_value,
                  resource_id='device001',
                  measurement='temperature',
                  sampling_rate=timedelta(seconds=60))

# Add event handling capability to the client
client.add_handler('on_event', handle_event)

# Run the client in the Python AsyncIO Event Loop
loop = asyncio.get_event_loop()
loop.create_task(client.run())
loop.run_forever()

You must use signal_name='LOAD_CONTROL'  and signal_type='x-LoadControlCapacity' (don't forget the x-).
Then, this function receives the event:
async def handle_event(event):
    # This callback receives an Event dict.
    # You should include code here that sends control signals to your resources.
    return 'optIn'

The event is a dictionary that contains the event information. It looks something like this:
{'active_period': {'dtstart': datetime.datetime(2021, 2, 1, 13, 0, 0, tzinfo=datetime.timezone.utc),
                   'duration': datetime.timedelta(minutes=60)},
 'event_descriptor': {'created_date_time': datetime.datetime(2021, 2, 1, 12, 0, 0, tzinfo=datetime.timezone.utc),
                      'event_id': '541c76cb-36db-4ece-bf06-1ec80d06aa5d',
                      'event_status': 'near',
                      'market_context': 'http://MarketContext1',
                      'modification_date_time': datetime.datetime(2020, 12, 3, 14, 22, 7, 606894, tzinfo=datetime.timezone.utc),
                      'modification_number': 0,
                      'priority': 1,
                      'test_event': False},
 'event_signals': [{'intervals': [{'dtstart': datetime(2021, 2, 1, 13, 0, 0, tzinfo=timezone.utc),
                                   'duration': timedelta(minutes=60),
                                   'signal_payload': 1.0}],
                    'signal_id': 'ca5a2b4b-69b4-40ee-93ca-dbfa23da545d',
                    'signal_name': 'LOAD_CONTROL',
                    'signal_type': 'x-loadControlCapacity'}],
 'response_required': 'always',
 'targets': [{'ven_id': 'ven_id_123'}],
 'targets_by_type': {'ven_id': ['ven_id_123']}}

From that dictionary, you should get all the info you need to send control signals to your device. The handle_event function should then return 'optIn' or 'optOut' to let the VTN know if you are taking part in the event.

Hi Stan,
Is there a print statement or anything needed in the handle_event function?
async def handle_event(event):
    # This callback receives an Event dict.
    # You should include code here that sends control signals to your resources.
    return 'optIn'

In my tests on the VEN side I am only seeing this information in the cmd prompt:
Adding 2021-02-02 16:19:00.022046+00:00, 69.09 to report
The number of intervals in the report is now 1
Report will be sent now.
69.09
Adding 2021-02-02 16:20:00.012119+00:00, 69.09 to report
The number of intervals in the report is now 1
Report will be sent now.
Execution of job "OpenADRClient._poll (trigger: cron[hour='*', minute='*', second='*/10'], next run at: 2021-02-02 10:20:09 CST)" skipped: maximum number of running instances reached (1)
Execution of job "OpenADRClient._poll (trigger: cron[hour='*', minute='*', second='*/10'], next run at: 2021-02-02 10:20:10 CST)" skipped: maximum number of running instances reached (1)
Execution of job "OpenADRClient._poll (trigger: cron[hour='*', minute='*', second='*/10'], next run at: 2021-02-02 10:20:30 CST)" skipped: maximum number of running instances reached (1)
69.09
Adding 2021-02-02 16:21:00.079957+00:00, 69.09 to report
The number of intervals in the report is now 1
Report will be sent now.
Execution of job "OpenADRClient._poll (trigger: cron[hour='*', minute='*', second='*/10'], next run at: 2021-02-02 10:21:20 CST)" skipped: maximum number of running instances reached (1)
68.23
Adding 2021-02-02 16:22:00.017048+00:00, 68.23 to report
The number of intervals in the report is now 1
Report will be sent now.
68.23
Adding 2021-02-02 16:23:00.054695+00:00, 68.23 to report
The number of intervals in the report is now 1
Report will be sent now.
67.9
Adding 2021-02-02 16:24:00.008960+00:00, 67.9 to report
The number of intervals in the report is now 1
Report will be sent now.
67.9
Adding 2021-02-02 16:25:00.061115+00:00, 67.9 to report
The number of intervals in the report is now 1
Report will be sent now.
67.9
Adding 2021-02-02 16:26:00.085833+00:00, 67.9 to report
The number of intervals in the report is now 1
Report will be sent now.

That process that I am doing is starting my BACnet read script, then my VTN script, and then my VEN script all on the same PC - localhost.

The separated BACnet and VEN scripts are sharing an SQLite db, seems to be working Ok.
I also added in the -x on the VTN script to get the signal type and name correct.

Also I know this is a Linux project the code seems to run on Windows too. Thought I noticed that experimenting around here...

Any tips on how to retrieve the event dictionary on the VEN side greatly appreciated. Just adding in a print to the handle_event function doesnt appear to show anything. Sorry if its super basic something I am missing!
async def handle_event(event):
    # This callback receives an Event dict.
    # You should include code here that sends control signals to your resources.
    print(event)
    return 'optIn'

Would this only show up if the event notification conditions is TRUE? For example if the future scheduled date & time event pushed down from the VTN to VEN == current date & time on the VEN. Hopefully that makes sense

The event should be right there as the event parameter, so you're doing the VEN part right.
This part in the your vtnTest.py file stands out to me:

# Add a prepared event for a VEN that will be picked up when it polls for new messages.
server.add_event(ven_id='ven_id_123',
                 signal_name='LOAD_CONTROL',
                 signal_type='x-LoadControlCapacity',
                 intervals=[{'dtstart': datetime(2021, 2, 1, 13, 0, 0, tzinfo=timezone.utc),
                             'duration': timedelta(minutes=60),
                             'signal_payload': 100.0}],
                 callback=event_response_callback)

It looks like you're adding an event that is already over. Events that are in the past won't be communicated by the VTN. You should make the dtstart parameter dynamic based on the current time probably:

```python3

# Add a prepared event for a VEN that will be picked up when it polls for new messages.
server.add_event(ven_id='ven_id_123',
                 signal_name='LOAD_CONTROL',
                 signal_type='x-LoadControlCapacity',
                 intervals=[{'dtstart': datetime.now(timezone.utc) + timedelta(minutes=5),
                             'duration': timedelta(minutes=60),
                             'signal_payload': 100.0}],
                 callback=event_response_callback)

Or something like that. Let me know if this fixes things for you.

Also:

Would this only show up if the event notification conditions is TRUE? For example if the future scheduled date & time event pushed down from the VTN to VEN == current date & time on the VEN. Hopefully that makes sense

The event will be communicated on the next poll cycle as long as it ends in the future. (With the exception of cancelled events which have their own rules, but that is not relevant here)

Stan thanks for the quick response, its super interesting to know these features. So I added in the intervals=[{'dtstart': datetime.now(timezone.utc) + timedelta(minutes=5),  and restarted the VTN script.
Watching the VEN side cmd prompt, I can see the HTTP 500 when the VTN script restarted but I still dont see the event dictionary.
Adding 2021-02-03 14:01:00.020965+00:00, 65.43 to report
The number of intervals in the report is now 1
Report will be sent now.
Non-OK status 500 when performing a request to http://localhost:8080/OpenADR2/Simple/2.0b/OadrPoll with data <?xml version="1.0" encoding="utf-8"?>
<oadr:oadrPayload xmlns:oadr="http://openadr.org/oadr-2.0b/2012/07">
<oadr:oadrSignedObject xmlns:oadr="http://openadr.org/oadr-2.0b/2012/07" oadr:Id="oadrSignedObject"><oadr:oadrPoll ei:schemaVersion="2.0b" xmlns:ei="http://docs.oasis-open.org/ns/energyinterop/201110"><ei:venID>ven_id_123</ei:venID></oadr:oadrPoll></oadr:oadrSignedObject>
</oadr:oadrPayload>: 500 500 Internal Server Error

Server got itself in trouble
65.43
Adding 2021-02-03 14:02:00.026755+00:00, 65.43 to report
The number of intervals in the report is now 1
Report will be sent now.
65.43
Adding 2021-02-03 14:03:00.084536+00:00, 65.43 to report
The number of intervals in the report is now 1
Report will be sent now.
Execution of job "OpenADRClient._poll (trigger: cron[hour='*', minute='*', second='*/10'], next run at: 2021-02-03 08:03:59 CST)" skipped: maximum number of running instances reached (1)
65.43
Adding 2021-02-03 14:04:00.110906+00:00, 65.43 to report
The number of intervals in the report is now 1
Report will be sent now.
Execution of job "OpenADRClient._poll (trigger: cron[hour='*', minute='*', second='*/10'], next run at: 2021-02-03 08:04:30 CST)" skipped: maximum number of running instances reached (1)
65.43
Adding 2021-02-03 14:05:00.020904+00:00, 65.43 to report
The number of intervals in the report is now 1
Report will be sent now.
65.43
Adding 2021-02-03 14:06:00.031671+00:00, 65.43 to report
The number of intervals in the report is now 1
Report will be sent now.
65.43
Adding 2021-02-03 14:07:00.070573+00:00, 65.43 to report
The number of intervals in the report is now 1
Report will be sent now.

Sorry if this is sort of large amount of stuff to post on a git issue but there is also a traceback on the VTN cmd prompt. The restart below is when I restarted the VTN script after adding in datetime.now(timezone.utc) + timedelta(minutes=5) on the VTN script
= RESTART: C:\Users\bbartling\OneDrive - Slipstream\Desktop\openLeadr\vtnTest.py

********************************************************************************
                       Your VTN Server is now running at
                   http://<IP_ADDRESS>:8080/OpenADR2/Simple/2.0b
********************************************************************************

Responding to oadrPoll with a oadrDistributeEvent message: {'events': [Event(event_descriptor=EventDescriptor(event_id='1f4d5e85-ac35-40e3-b18e-11d288936937', modification_number=0, market_context='oadr://unknown.context', event_status='far', created_date_time=datetime.datetime(2021, 2, 3, 14, 0, 55, 176008, tzinfo=datetime.timezone.utc), modification_date_time=datetime.datetime(2021, 2, 3, 14, 0, 55, 176008, tzinfo=datetime.timezone.utc), priority=0, test_event=False, vtn_comment=None), event_signals=[EventSignal(intervals=[{'dtstart': datetime.datetime(2021, 2, 3, 14, 5, 55, 174972, tzinfo=datetime.timezone.utc), 'duration': datetime.timedelta(seconds=3600), 'signal_payload': 100.0}], signal_name='LOAD_CONTROL', signal_type='x-LoadControlCapacity', signal_id='b975b53b-a98e-4391-a860-de8f4b673d46', current_value=None, targets=None, targets_by_type=None, measurement=None)], targets=[{'ven_id': 'ven_id_123'}], targets_by_type={'ven_id': ['ven_id_123']}, active_period=ActivePeriod(dtstart=datetime.datetime(2021, 2, 3, 14, 5, 55, 174972, tzinfo=datetime.timezone.utc), duration=datetime.timedelta(seconds=3600), tolerance=None, notification_period=None, ramp_up_period=None, recovery_period=None), response_required='always')], 'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'bc340026-6cae-475b-b1f9-043086f4e129'}.
Error handling request
Traceback (most recent call last):
  File "C:\Users\bbartling\AppData\Local\Programs\Python\Python37\lib\site-packages\aiohttp\web_protocol.py", line 422, in _handle_request
    resp = await self._request_handler(request)
  File "C:\Users\bbartling\AppData\Local\Programs\Python\Python37\lib\site-packages\aiohttp\web_app.py", line 499, in _handle
    resp = await handler(request)
  File "C:\Users\bbartling\AppData\Local\Programs\Python\Python37\lib\site-packages\openleadr\service\vtn_service.py", line 126, in handler
    msg = self._create_message(response_type, **response_payload)
  File "C:\Users\bbartling\AppData\Local\Programs\Python\Python37\lib\site-packages\openleadr\messaging.py", line 83, in create_message
    validate_xml_schema(ensure_bytes(msg))
  File "C:\Users\bbartling\AppData\Local\Programs\Python\Python37\lib\site-packages\openleadr\messaging.py", line 93, in validate_xml_schema
    tree = etree.fromstring(content, XML_PARSER)
  File "src\lxml\etree.pyx", line 3237, in lxml.etree.fromstring
  File "src\lxml\parser.pxi", line 1896, in lxml.etree._parseMemoryDocument
  File "src\lxml\parser.pxi", line 1784, in lxml.etree._parseDoc
  File "src\lxml\parser.pxi", line 1141, in lxml.etree._BaseParser._parseDoc
  File "src\lxml\parser.pxi", line 615, in lxml.etree._ParserContext._handleParseResultDoc
  File "src\lxml\parser.pxi", line 725, in lxml.etree._handleParseResult
  File "src\lxml\parser.pxi", line 654, in lxml.etree._raiseParseError
  File "<string>", line 0
lxml.etree.XMLSyntaxError: Element '{http://docs.oasis-open.org/ns/energyinterop/201110}signalType': [facet 'enumeration'] The value 'x-LoadControlCapacity' is not an element of the set {'delta', 'level', 'multiplier', 'price', 'priceMultiplier', 'priceRelative', 'setpoint', 'x-loadControlCapacity', 'x-loadControlLevelOffset', 'x-loadControlPercentOffset', 'x-loadControlSetpoint'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'fb9d3a57-a882-442a-a158-346982788af8'}.
You should implement and register your own on_update_report handler to deal with reports that your receive from the VEN. This handler will receive either a complete oadrReport dict, or a list of (datetime, value) tuples that you can then process how you see fit. You don't need to return anything from that handler.
Responding to oadrUpdateReport with a oadrUpdatedReport message: {'vtn_id': 'myvtn', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '35b178e0-b38e-4b9c-a39a-b71699d65510'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '12f2f5f7-25b0-4644-97f4-f76c99aa5de6'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '464a8c68-2d11-4601-bfbc-3f22d65bd786'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '17821c60-ba13-4fcc-af0e-d2bf107c1f83'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'b16e881e-4112-4655-8cc7-83d15f5c9b18'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '072804a1-31a9-4521-989d-25c88bcc93a9'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '8cb4d3f9-9102-40d5-a07e-5288b63014c4'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '0f796e95-f718-4805-9eb5-9a03dd5c25dd'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '27eb5a8f-698f-44c5-b711-2422c196cda0'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '6c73c8ae-9878-49e9-9d25-7d8bd0a78c22'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'cfa27b1f-f3e8-478a-b8db-5eca9361fb22'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'd032fe64-a28a-4084-860e-e0a7b5eb1296'}.
You should implement and register your own on_update_report handler to deal with reports that your receive from the VEN. This handler will receive either a complete oadrReport dict, or a list of (datetime, value) tuples that you can then process how you see fit. You don't need to return anything from that handler.
Responding to oadrUpdateReport with a oadrUpdatedReport message: {'vtn_id': 'myvtn', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '8d402255-5a53-4d34-bd7c-def53ace72b4'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'ed26d49b-fa54-485a-a28d-f7206d184994'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'ca18dc45-54fa-4bb7-8274-0ba9d36806ad'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '84c0337a-ff73-4dd6-ba1a-8cfe882f4cfb'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '41412f62-2c89-4d41-889e-867e1a5aab98'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '94852051-d63e-4473-86c9-b63aa972e1ad'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '4e3508c7-e8ea-4eb7-9e0b-6b18fd4c1f90'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '10d3b915-51eb-430d-865a-cdcc34aa3722'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '15203a73-53c0-461b-9a96-cdc33b2d7153'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '2b2e5d54-a05f-48f7-96c2-729b69ce4a0f'}.
You should implement and register your own on_update_report handler to deal with reports that your receive from the VEN. This handler will receive either a complete oadrReport dict, or a list of (datetime, value) tuples that you can then process how you see fit. You don't need to return anything from that handler.
Responding to oadrUpdateReport with a oadrUpdatedReport message: {'vtn_id': 'myvtn', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '27770e06-e4dd-463b-9847-f73651920814'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '4424c68a-e107-44ac-8eb9-f62b6f71d2b3'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '9f174387-f967-43c5-bdfa-afe2ec807f8c'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'dcffcd58-4730-4bde-b9db-6ba159e0e6b5'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'ceb1ee5d-e82e-473b-804f-023779c22f47'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'b4a06d29-04ab-40df-a3d2-02ee69cebd05'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'a87fab48-07ff-4e35-a5bb-1ebc501b49ca'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '4f12060f-7605-4218-9a01-d8bca54fd5ed'}.
You should implement and register your own on_update_report handler to deal with reports that your receive from the VEN. This handler will receive either a complete oadrReport dict, or a list of (datetime, value) tuples that you can then process how you see fit. You don't need to return anything from that handler.
Responding to oadrUpdateReport with a oadrUpdatedReport message: {'vtn_id': 'myvtn', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '612cbb1d-cbcf-4cee-91cb-62e448eb6360'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '07d8e3b8-8047-4479-a959-fc82e02b7c84'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'd7d0c529-21b2-427c-baf8-361c15bbfffb'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '94d547fb-355b-470d-9302-908ac4ed1c48'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '14dcbe96-771c-43a2-a71e-6431e92a6cf9'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '3d96587e-d87c-4fbd-9fb3-b5d0ec6f51f9'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '0f3814a7-a848-46c1-8d36-86e89276dbe8'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '900e7cd8-de73-4238-b5b0-6afaae7b62c2'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'd01f128f-6009-46bb-85af-c05ccae7901f'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'd108f433-b3ac-4013-9440-48eff6469c2d'}.
You should implement and register your own on_update_report handler to deal with reports that your receive from the VEN. This handler will receive either a complete oadrReport dict, or a list of (datetime, value) tuples that you can then process how you see fit. You don't need to return anything from that handler.
Responding to oadrUpdateReport with a oadrUpdatedReport message: {'vtn_id': 'myvtn', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'd0ea3959-0b1d-464e-9c79-ce5867394991'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '298f66d8-8e1b-4b53-b8b0-3f94f2b727d1'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'b9eee739-e563-4298-9cd6-4b5bc551e21f'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '27cc61c3-4782-44b7-a6cc-27e2ee26e9f6'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'e00b3d6a-b05f-4b9b-ac79-e1e50c86dc32'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '8b7e16fd-af78-46d8-83af-042d19de15b9'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '306d72ab-e218-41ee-8eb5-68d5321e1012'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '5b11787f-2699-4c78-b232-fcda43165a6b'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'f8eb4982-a793-4173-9244-20c8ef74cb14'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'b83d4f12-ebfc-44c9-8305-81354bbe1675'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '267a655c-cf9b-4e2d-8716-1ac275d597b0'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'aea3b587-d0a9-488a-bcf1-fc071661169d'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '8ead8f2e-0640-48f2-ab53-4573c0dfd9ac'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '09bf7174-ac60-4c79-af93-ba09516712e0'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '1c54f378-804e-460c-9075-f2b6bf2aa921'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '34438920-1a46-4fbc-b4b2-1836501efccb'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '685698f2-59f1-4d1c-a456-a9461cd3d322'}.
You should implement and register your own on_update_report handler to deal with reports that your receive from the VEN. This handler will receive either a complete oadrReport dict, or a list of (datetime, value) tuples that you can then process how you see fit. You don't need to return anything from that handler.
Responding to oadrUpdateReport with a oadrUpdatedReport message: {'vtn_id': 'myvtn', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '447a2c6d-c545-4aeb-8f1d-709270b67abe'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'bfdc31bc-5d44-4e8b-9b5e-ab9c673b62bf'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'd435b2da-ef35-452b-8a22-3cba20449215'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '824cb905-58bf-44b6-97ba-f8324c9da12b'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '6ad815fb-6b64-4a34-ac0a-a1004363bb86'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '9995d8fe-d1f0-4907-9ba2-c39492f1aba1'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '503ca5ac-5179-48a7-bd48-91f1a791e416'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '3f2d2171-72dd-4401-994f-b509f6c800a4'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'de2d016b-c291-46c3-b91d-a0ea2e2277b7'}.
You should implement and register your own on_update_report handler to deal with reports that your receive from the VEN. This handler will receive either a complete oadrReport dict, or a list of (datetime, value) tuples that you can then process how you see fit. You don't need to return anything from that handler.
Responding to oadrUpdateReport with a oadrUpdatedReport message: {'vtn_id': 'myvtn', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '1ab083dd-075a-4e7a-865e-8c401f4b3a53'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '3fcf875c-05fb-4cbc-aee1-744bdcb15c8e'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '3d27eb33-983a-43b7-8f94-944036fd2bcf'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'ed9e21f7-6d6e-48c1-aa87-52d0e3dbbe3c'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '7ebeb6e2-08c4-456f-80ec-e880477e3ff6'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': 'd9ec338f-805b-4b7a-842f-277b2b891416'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '46f61ca7-8a02-4ab7-a1f8-ceed5a093ee5'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '5607ed32-a84e-488e-bb1b-bafc4d16fbec'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '60f8b879-2114-4eb3-9d43-eaa95af20126'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '3e3c5641-a715-41da-94b8-35dfac6bb175'}.
Responding to oadrPoll with a oadrResponse message: {'vtn_id': 'myvtn', 'ven_id': 'ven_id_123', 'response': {'request_id': None, 'response_code': 200, 'response_description': 'OK'}, 'request_id': '487a5120-3ea7-4301-9601-f3328a1d2fd6'}.
You should implement and register your own on_update_report handler to deal with reports that your receive from the VEN. This handler will receive either a complete oadrReport dict, or a list of (datetime, value) tuples that you can then process how you see fit. You don't need to return anything from that handler.

One more comment sorry!
I am re-watching the Linux foundation YouTube video this is what I need to experiment with when you demonstrated this:

So the problem with that HTTP 500 error on the VTN is this:
lxml.etree.XMLSyntaxError: Element '{http://docs.oasis-open.org/ns/energyinterop/201110}signalType': [facet 'enumeration'] The value 'x-LoadControlCapacity' is not an element of the set {'delta', 'level', 'multiplier', 'price', 'priceMultiplier', 'priceRelative', 'setpoint', 'x-loadControlCapacity', 'x-loadControlLevelOffset', 'x-loadControlPercentOffset', 'x-loadControlSetpoint'}.

You have to fix the capitalization of x-loadControlCapacity. The capital L must be a lowercase. To avoid these problems, you can use the openleadr.enums.SIGNAL_TYPE.X_LOAD_CONTROL_CAPACITY. I will also add a check to give a more friendly error message instead of this ugly mess ;-)

For experimentation, I sometimes find it handy to set a breakpoint() in the event handler, so that you can inspect the event while it is running when you're developing. If you're unfamiliar, once Python hits the breakpoint(), it will drop into an interactive python console that you can use to do things. If you want to get out again, you type continue and hit enter.

async def handle_event(event):
    breakpoint()
    ...

Cool thanks Stan no I am not familiar with the breakpoint will look more into this information :)

Hi Stan,
So when I fixed the VTN side with the l being lowercase the event dictionary did pop up on the VEN side.
>>>
= RESTART: C:\Users\bbartling\OneDrive - Slipstream\Desktop\openLeadr\venTest_sql_async.py
VEN is now registered with ID ven_id_123
The polling frequency is 0:00:10
{'event_descriptor': {'event_id': '7b52396e-296a-459b-98e9-efaea16b8bbf', 'modification_number': 0, 'modification_date_time': datetime.datetime(2021, 2, 4, 15, 43, 1, 305317, tzinfo=datetime.timezone.utc), 'priority': 0, 'market_context': 'oadr://unknown.context', 'created_date_time': datetime.datetime(2021, 2, 4, 15, 43, 1, 305317, tzinfo=datetime.timezone.utc), 'event_status': 'far', 'test_event': False}, 'active_period': {'dtstart': datetime.datetime(2021, 2, 4, 15, 48, 1, 304340, tzinfo=datetime.timezone.utc), 'duration': datetime.timedelta(seconds=3600)}, 'event_signals': [{'intervals': [{'dtstart': datetime.datetime(2021, 2, 4, 15, 48, 1, 304340, tzinfo=datetime.timezone.utc), 'duration': datetime.timedelta(seconds=3600), 'uid': 0, 'signal_payload': 100.0}], 'signal_name': 'LOAD_CONTROL', 'signal_type': 'x-loadControlCapacity', 'signal_id': '6a186e86-c928-429c-8a5e-c0dca718b312'}], 'targets': [{'ven_id': 'ven_id_123'}], 'targets_by_type': {'ven_id': ['ven_id_123']}, 'response_required': 'always'}
oadrResponse
65.43
Adding 2021-02-04 15:44:00.068109+00:00, 65.43 to report
The number of intervals in the report is now 1
Report will be sent now.
65.43
Adding 2021-02-04 15:45:00.018330+00:00, 65.43 to report
The number of intervals in the report is now 1
Report will be sent now.
65.43
Adding 2021-02-04 15:46:00.029211+00:00, 65.43 to report
The number of intervals in the report is now 1
Report will be sent now.
65.43
Adding 2021-02-04 15:47:00.033336+00:00, 65.43 to report
The number of intervals in the report is now 1
Report will be sent now.
65.43
Adding 2021-02-04 15:48:00.019989+00:00, 65.43 to report
The number of intervals in the report is now 1
Report will be sent now.
65.43

Does the event dictionary only show up once? Which I am assuming came from the print on my handle_event(event) function on VEN script?
async def handle_event(event):
    # This callback receives an Event dict.
    # You should include code here that sends control signals to your resources.
    print(event)
    return 'optIn'

I am just day dreaming how this could be linked up to a BACnet write command for control when the DR event is True and BACnet release to Auto when the DR event is False

The event dict only shows up once, which is when it is created. It can also show up if the event is ever updated; it will then be handed off to your 'on_update_event' handler (which works the same way as the 'on_event' handler).
The relevant part of the code for the demo VEN in that video was this:
...
from gpiozero import Button, LED

SWITCH = LED(2)
SWITCH.off()
STATUS = 0
def get_status():
    return STATUS

async def change_status(status, delay):
    """
    Change the switch position after 'delay' seconds.
    """
    global STATUS
    if delay > 0:
        await asyncio.sleep(delay)
    if status == 1 and STATUS == 0:
        print("SWITCHING ON")
        SWITCH.on()
    elif status == 0 and STATUS == 1:
        print("SWITCHING OFF")
        SWITCH.off()
    STATUS = status

async def handle_event(event):
    """
    Do something based on the event.
    """
    loop = asyncio.get_event_loop()
    signal = event['event_signals'][0]
    intervals = signal['intervals']
    now = datetime.now(timezone.utc)
    for interval in intervals:
        start = interval['dtstart']
        delay = (start - now).total_seconds()
        value = interval['signal_payload']
        print(f"Setting status {value} after {delay} seconds.")
        loop.create_task(change_status(status=int(value),
                                       delay=delay))
    return 'optIn'

In this case, I used a signal with name 'simple' and type 'level' (which is an integer: 0, 1, 2 or 3). I mapped these to 0 = off and 1 = on. You can see that I scheduled the event to start on the right moment by adding a delay to the switching.
Obviously, this is very simple code that was just used to demo the principles of the openADR system, but hopefully it is useful to you. The full VEN code that ran during that demo is in a gist here: https://gist.github.com/stan-janssen/3162b58047a476157a3f9e033522da1a

Cool thanks Stan for sharing this info.
One other question before closing this issue, (I think I am good to go for the moment) would you ever imagine a future GUI on the VTN side? Or some method of being able to create/update DR events without having to rerun the script?
This Java version of the openADR, I think this is what they are doing with rabbitMQ, but not entirely sure. And I know next to nothing about rabbitMQ, MQTT, or zeroMQ; I was curious if one of these (or some rest API) would be something worth while experimenting with? If you have any incite on preferred methods/packages, any tips that would work well with openLeadr greatly appreciated :)
Hopefully this makes sense but on the VTN side it would be how the utility provider or aggregator would update DR events.
In here on the VTN script: dtstart, duration, signal_payload
# Add a prepared event for a VEN that will be picked up when it polls for new messages.
server.add_event(ven_id='ven_id_123',
                 signal_name='LOAD_CONTROL',
                 signal_type='x-loadControlCapacity',
                 intervals=[{'dtstart': datetime.now(timezone.utc) + timedelta(minutes=5),
                             'duration': timedelta(minutes=60),
                             'signal_payload': 100.0}],
                 callback=event_response_callback)

One other question before closing this issue, (I think I am good to go for the moment) would you ever imagine a future GUI on the VTN side?

A demo VTN with a GUI is in the works, keep your eye on the openleadr-example-vtn repository. I'm still working on it, but it looks something like this at the moment:

Or some method of being able to create/update DR events without having to rerun the script?

The server.add_event can be called at any time, also after starting the server. This is what I did in the VTN from the demo video; take a look at the gist. I imagine you can create some connection to an event bus (like RabbitMQ), or perhaps some internal REST API endpoint, or some polling mechanism, which then creates the events in your VTN code.
So yes to both of your questions. The GUI is really only meant as a demonstration thing where you can see what's going on and to see a built-out implementation. Creating events is really something you want to automate, which is already possible with the OpenLEADR VTN as it stands right now.

Wow, cool. Will do looks great.
I got it that makes sense now. The control logic is in the VTN script.
One thing I am curious about is knowing more how utility providers operate (grid operator control room) where I think they use forecasting techniques to see next day potential grid issues. (sub station/load forecasting analytics, etc.)
(more research on my end) But if there is some other software platform for grid operators to see next day grid issues at a certain window of time (like tomorrow afternoon when its hot grid is stressed from 1PM-4PM) will it be possible through the new GUI to set future date ranges like almost a calendar look and feel?
From the utility providers we work with they all seem very old fashion still relying on phone calls to the building for DR. I have a feeling Europe maybe further along in open ADR :)

ADR in Europe is also still just building up. We have the regular reserves for balancing the overall electricity grid, but that's all managed electrically (rather than electronically): the reserve units measure the grid frequency and regulare up and down automatically to correct the grid frequency that way (a cool realtime visualization is here: https://www.mainsfrequency.com/).
At the moment, the GUI just has support for two predefined 'dummy' events for testing purposes. It is certainly possible to add an interface for scheduling events, but that would get quite complicated (you have to set the signal name and type, the list of targets, which can be of varying types, and then add the intervals for the signal, each with their own start date or interval, and signal payload).
In the OpenLEADR VTN, you can easily add your own REST endpoints for receiving 'push' events from which you can create your own events, or you can set up a polling task that will hit a REST endpoint every X seconds/minutes.
OpenLEADR makes use of aiohttp under the hood, so you can find more information about it in their documentation. The aiohttp 'app' is located at server.app, so adding your own route would look something like this:

from openleadr import OpenADRServer
from aiohttp import web

async def event_api(request):
    """
    This gets called when a POST request is made to `/event_api`.
    """
    # Load the JSON data in the REST call
    data = await request.json()

    # Create an event based on the information in the REST call
    server.add_event(ven_id=data['ven_id'],
                     signal_name=data['signal_name'],
                     ...)

server = OpenADRServer(...)
server.app.add_routes([web.post('/event_api', event_api)])
...

(This reference to server in the event_api is a little fragile right now, so I'm going to make it so that you can refer back to the server from the app variable itself, like request.app['server'].add_event(...)), so that you can have your event_api() code somewhere other than where you define your server instance).
With the above code, after you start the server, you now have an extra endpoint at http://localhost:8080/event_api, where you can POST a json object to, which then gets turned into an event.
You could also set up a polling task. A simple way to do it would be like this:

import aiohttp
from openleadr import OpenADRServer

server = OpenADRServer(...)

...

async def poll_for_new_events(server):
    """
    Hit a REST endpoint every 10 seconds to see if we need to create an event
    """
    # Create an aiohttp ClientSession (see their docs for more info)
    client_session = aiohttp.ClientSession()

    # Keep running this loop
    while True:
        # Hit the REST endpoint that tells us if we need to take action
        with client_session.post("http://some-server.com:9090/events") as req:
            # Load the returned data
            data = await req.json()
            # Inspect the data, and create an event if necessary
            if data['shit_hits_the_fan'] is True:
                server.add_event(...)
        # Wait for 10 seconds until the next check, without blocking the event loop
        await asyncio.sleep(10)

loop = asyncio.get_event_loop()
loop.create_task(server.run())
loop.create_task(poll_for_new_events(server))

Hey thanks Stan, this is pretty cool. I have tinkered around with a few other platforms and this has been quite the easiest one to work with right from the VTN/VEN examples scripts. I was actually somewhat shocked it worked right from the initial try, and its on PyPi which is cool too.
I'm going to close this for now, more to come we will be doing alot of GEB test bed project work for open ADR this coming summer as our company office as a test bench. Thanks for the code you showed me above, very inspirational :)


Geofence Region Monitoring

Hi,
Does SwiftLocation project supports Geofence Region monitoring? OR
How to achieve below functionality using SwiftLocation project?
let cRegion = CLCircularRegion(center: location.coordinate, radius: 180.0, identifier: "identifer")
cRegion.notifyOnEntry = true
cRegion.notifyOnExit = true
locationManager.startMonitoring(for: cRegion)
Thanks & Regards,
Pravin

Yes since 5.0 we support geofencing. take a look at the doc.

@malcommac  Do you still stand by this? custom polygon region (actually it does not work correctly so avoid it) 🤔


Dragging an unfocused field

Previously, it wasn't really possible to unfocus a field during a drag gesture; the field would be immediately re-focused at the next dragupdate and the selection changes would continue.  This PR stops listening to drag events when the field is not focused.
Fixes https://github.com/flutter/flutter/issues/93143

Does this prevent edge scrolling if the text field yields focus once the touch point goes outside of its bounds?

I've just merged in master so we get the edge scrolling PR.  The answer is yes, but I think the behavior is desired for that case, even if it is a little weird.
Without the MouseRegion that unfocuses on exit, the edge scrolling behavior is the same with/without this PR.
Here is a video of me testing edge scrolling on this PR using the the example app from the issue https://github.com/flutter/flutter/issues/93143.
https://user-images.githubusercontent.com/389558/142516067-9f508090-dfdb-4e9b-adb6-9ef28bc3e039.mov

Gold has detected about 1 new digest(s) on patchset 3.
View them at https://flutter-gold.skia.org/cl/github/93333

Gold has detected about 7 new digest(s) on patchset 4.
View them at https://flutter-gold.skia.org/cl/github/93333

@LongCatIsLooong Ready for re-review.

Gold has detected about 7 new digest(s) on patchset 9.
View them at https://flutter-gold.skia.org/cl/github/93333

Talking to @LongCatIsLooong, it seems like this PR needs a bit of refactoring to be able to land with the correct behavior.  The tap gesture detector in RenderEditable prevents us from being able to set focus in the gesture detectors.  We need to move that out of RenderEditable before this solution can move forward.
The goal is still the behavior agreed upon above.
I'm going to put this PR back into draft mode.

(triage): Are there still plans for this PR?

Sorry I abandoned this PR for awhile.  I'll get it up to date and then either finish it or close it for now depending on how relevant it still looks.

I'm going to close this as I investigate refactoring TextSelectionControls as a part of my context menu work.  Once that is done, I should reinvestigate if it makes this refactor easier.


cmd/go/internal/load: filepath.Split used on a '/'-separated path

On master right now:
https://github.com/golang/go/blob/1024b25d0c044484ed9288ac8f78bd9f5ebb5287/src/cmd/go/internal/load/pkg.go#L1192
Import path is always '/'-separated, but filepath.Split operates on OS-defined file paths that may not be '/'-separated.
This was introduced as part of CL 140863, I don't think it was intentional. See my comment there for details:
https://go-review.googlesource.com/c/go/+/140863/11#message-db0ff6bb2c7b06161ca47de771c4465afa8b1102
/cc @hyangah @bcmills @bradfitz

For posterity, I made CL 167618 where I intentionally simulated what would happen on an operating system where forward slash (/) is not a recognized filepath separator, to see if there are any tests to catch that.
It failed because the go binary did not get installed with the right name (it had a name like cmd/go.exe instead of just go.exe), so it didn't get very far:
Building Go cmd/dist using C:\workdir\go1.4
Building Go toolchain1 using C:\workdir\go1.4.

Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for windows/amd64.
go tool dist: FAILED: C:\workdir\go\bin\go list -gcflags=all= -ldflags=all= -f={{if .Stale}}	STALE {{.ImportPath}}: {{.StaleReason}}{{end}} std cmd: exec: "C:\\workdir\\go\\bin\\go": file does not exist
The system cannot find the batch label specified - fail

Error: build failed: make script failed: exit status 1

(Source: https://storage.googleapis.com/go-build-log/a564c9ab/windows-amd64-2016_1e739c43.log.)


Generic type inference failure when multiple type provided (with typescript)

Writing below code via ES5 with commonjs,
Rx.Observable.of(1, 'a').subscribe(console.log);
will be executed with below results

1
a

though some editors might complain by type inference failure.
if same code is written in typescript, now those complaint becomes build time failure

error TS2453: The type argument for type parameter 'T' cannot be inferred from the usage. Consider specifying the type arguments explicitly.
Type argument candidate 'number' is not a valid type argument because it is not a supertype of candidate 'string'.

Somewhat related to issue https://github.com/ReactiveX/RxJS/issues/581, not directly though.

I'm looking into this to try resolve.

Course caller can explicitly cast like
Rx.Observable.of(<any>1,<any>'a').subscribe(console.log);
, but seems bit hassle.

Can you write Rx.Observable.of<any>(1, a').subscribe(console.log); instead?

Yes, that works too. Snippet I took is exaggerated examples of casting hassle. Still it'd be better wouldn't require explicit casting, not sure if it's achievable along with generic based type-safety.

I don't think that Rx.Observable.of<any>(1, 'a') is a hassle, IMO it's perfectly reasonable TypeScript. It also makes you think whether you really want to mix types.

Ok, closing now.


Please check all of the platforms you are having the issue on (if platform is not listed, it is not supported)

[ ] iOS
[x] Android
[ ] UWP
[ ] watchOS
[ ] tvOS
[ ] Tizen
[ ] .NET Standard

Feature Library

[ ] Core
[x] BluetoothLE
[ ] Beacons
[ ] Locations
[ ] HTTP Transfers
[ ] Notifications
[ ] Sensors

Version of Library
<IP_ADDRESS>3-beta
Version of OS(s) listed above with issue
Android 10
Steps to Reproduce

Use BLE central and peripherals
Load connected peripherals
check peripherals.context.Gatt

Expected Behavior
Gatt beeing not null or having an exception thrown
Actual Behavior
Gatt is null, no exception is thrown and further calls like peripheral.DiscoverServices() will fail

Load services
services.UseBleCentral();
services.UseBlePeripherals();

Load connected peripherals
_connectedDisposable = _centralManager.GetConnectedPeripherals().Subscribe(scanResult =>
            {
                Console.WriteLine("GetConnectedPeripherals results");
                scanResult.ToList().ForEach(
                 item =>
                 {
                     Console.WriteLine(item.Name);
                     if (!string.IsNullOrEmpty(item.Name))
                     {
                         var vm = new PeripheralItemViewModel(item);
                         Peripherals.Add(vm);
                     }
                 });

                _connectedDisposable?.Dispose();
            });

            if (_centralManager.IsScanning)
                _centralManager.StopScan();

You aren't using the library properly at all.  You don't touch the context or the gatt object directly.  You should wait until the docs are done or look at the sample code before using this library

I did not intend to use the Gatt object directly. I just got an exception here  and I noticed, that the Gatt property was null.
I did check the sample code, but I guess I'll wait untill docs are done.

Seems to be an Android related problem with this.NativeDevice.connectGatt

@twsl a few things - this module is in beta.

You are welcome to submit PRs
Your original issue is not an issue.  You still have to issue a connection to the GATT.  Please do not keep posting on a closed issue
The docs are not done as I’ve said.  I’m one person and this project is quite large.
This is not a support form.  The code is open for you to participate in fixing and benefiting from the end result


Documentation of WebApp Developments Milestone

Exercise:
Document the files where changes were made
• Using PlantUML diagrams
• And JSDoc


feat: enforce user text preferences without audio concern

Description

Enforce user text preferences without taking audio into account. An user could like to have French audio, with French subtitles (or captions for hearing impaired) at the same time, who are we to judge ? 😄 My use case is that we save what the user chose as audio / text languages, and then we always start with those languages (and / or roles)
I tried to make some archeology on the codebase to see why the feature was developed like that, but I wasn't able to find an explanation.
Screenshots (optional)

Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to
not work as expected)
[x] This change requires a documentation update

Checklist:

[x] I have signed the Google CLA https://cla.developers.google.com
[x] My code follows the style guidelines of this project
[x] I have made corresponding changes to the documentation
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have verified my change on multiple browsers on different platforms
[x] I have run ./build/all.py and the build passes
[x] I have run ./build/test.py and all tests pass

@joeyparrish 👋 WYT of this PR ?

Hey @theodab, I must concede you bested me in term of archeology 😄
Well in our Shaka-Player usage, we struggled to understand why our preferredTextLanguage / preferredTextRole was never applied and that was causing it, I came to think that we are not the only ones (ok, we could have read the doc)
I guess if this is properly advertised in the changelog of the release this commit is going into, that wouldn't be a problem !
I think the "only" thing needed to get the previous behaviour is to do something like:
preferredAudioLanguage: audioLanguage,
preferredTextLanguage: textLanguage === audioLanguage ? '' : textLanguage,

Let me know if you want to go through with it, or if I should add a new configuration key to keep the existing behaviour :)

I suppose I don't understand the problem.  Perhaps I misunderstand or misremember the state of the player, but when I read this:

Enforce user text preferences without taking audio into account. An user could like to have French audio, with French subtitles (or captions for hearing impaired) at the same time, who are we to judge ? 😄 My use case is that we save what the user chose as audio / text languages, and then we always start with those languages (and / or roles)

I feel like this is a non-issue from my perspective.  An application may:

Set a preferred audio language and text language in the player configuration to influence the initial choices of the player
Choose specific audio or text languages after calling load() to override those initial choices
Choose to enable text display before or after calling load()
Choose to disable auto-text display after calling load()

It looks to me like you're changing the auto-text display rules with this PR, but I have a hard time connecting that to how I understood your goal:

An user could like to have French audio, with French subtitles (or captions for hearing impaired) at the same time, who are we to judge ?

We don't judge the choice!  If you ask for those via preference or via select*Track() calls, we don't override the application.  Those will be the selected tracks.

My use case is that we save what the user chose as audio / text languages, and then we always start with those languages (and / or roles)

I don't see how the automatic decision to enable text breaks this for you.  We don't override the choice of languages at all, we just choose based on the languages whether or not text should be enabled by default.  The alternative would be to require apps to enable text streaming/display explicitly in addition to selecting languages or tracks.
Currently, the automatic decision to enable text is if:

It's not already enabled by the app and
The text track matches the text preference and
The audio track doesn't match the text track

So if you want to always start with the user's preferred text language, you would:

Configure the audio & text preferences
Call setTextTrackVisibility(true) on the player to enable text streaming and display
Call load() on the player

If the user has no text preference, don't configure one and don't call setTextTrackVisibility(true), and we won't stream or display text.
It's possible I misunderstood you.  If so, please forgive me and try to explain it to me so I can understand better.
It's also possible that app developers would prefer not having any such automatic choices made like this.  For example, we used to automatically disable ABR when someone selected an explicit track, but developers complained that this implicit behavior was surprising.  So we changed it, and now if you want to select a specific track and not have it changed by ABR, you have to also explicitly disable ABR.  Perhaps auto-enabling text display is one of these things that is surprising and confusing, and we would be open to feedback and possibly removing it.  Please let us know if this is the case!
However, just removing it would break compatibility, which we wouldn't do until we're ready for Shaka Player v4.  Before that, we would need an opt-in config to disable the auto-enabling of text streams, and possibly a warning log about the coming change to give developers time to adjust.

I think the "only" thing needed to get the previous behaviour is to do something like:
preferredAudioLanguage: audioLanguage,
preferredTextLanguage: textLanguage === audioLanguage ? '' : textLanguage,

Let me know if you want to go through with it, or if I should add a new configuration key to keep the existing behaviour :)

That's not quite what we do. We actually ignore the region-component of the language, so that (for example) we consider fr-CA and fr to be equivalent, and don't show French Canadian people French subtitles (or US English-speaking people English subtitles, etc). That a case that some developers might not think about, and could lead to unexpected problems when updating to the newest Shaka Player version.

👋
I think I wasn't that clear in my first explanation.
I have a stream with the following tracks:

audio

A1: fre
A2: eng

text

T1: fre (role subtitle)
T2: fre (role caption)

I set the following configuration through Shaka configure:
preferredAudioLanguage: 'fre',
preferredTextLanguage: 'fre',
preferredTextRole: 'caption'

When I load the stream, I was expecting to see A1 and T2 to be automatically selected and shown to the user.
Instead, because preferredTextLanguage is the same as preferredAudioLanguage, Shaka starts only with A1 and not showing the text track.
Set a preferred audio language and text language in the player configuration to influence the initial choices of the player
Choose specific audio or text languages after calling load() to override those initial choices
Choose to enable text display before or after calling load()
Choose to disable auto-text display after calling load()

So If I understand what you are saying, while wanting to enforce a user preference, I should write a block a code to get the actual textTracks of the stream, and see if any matches my actual user preference (and write a specific code to ignore region-component @theodab is speaking about) to be able to call / or not setTextTrackVisibility(true).
Auto-enabling textTracks is alright for me, what was surprising is the implicit choice of not auto-displaying user preferences when configured.

@joeyparrish 👋 Any thoughts on my previous message ? Thanks :)

In retrospect, it seems that the logic to auto-enable text is somewhat magical and confusing.  Perhaps the best thing is just to remove it.  The selected text and audio tracks would be chosen based on preferences, but text wouldn't be shown unless requested explicitly.  It would be up to the app to look at tracks, and create their own logic for enabling based on user preferences.
Alternately, we could have an explicit config to choose between the old and new behavior (boolean).
Yet another option would be to have an enum-based config to choose behavior for auto-enabling text:

legacy - auto-enable only if the text matches user prefs and audio differs
on text match - auto-enable only if the text matches user prefs
always - auto-enable text if any text is available
never - text must be explicitly enabled by the app

This last option could be overkill, but would give both maximum flexibility and backward compatibility.
@valotvince, thoughts?

Hi @joeyparrish ,
From my point of view, it is better to have the text preferences do their job (auto-enabled if existing), let the app change the text track by itself and not having some magic happening when switching of audio track. That being said, having the audio being taken into account could answer to some use cases (that I am not familiar with).
The text track handling is something we completely manage in our app because of Shaka today's behaviour, and we specifically don't use the text track prefs.
If you're okay with it I could rework that PR having:

a configuration switch between old & "new" behaviour => the switch would be on the old behaviour per default
keep the existing PR code for the "new" behaviour

@valotvince, sorry for the delay in my reply.  That sounds like a great plan.  Thanks!

@joeyparrish @avelad At last had the opportunity to rewrite the PR according to reviews. Let me know what you think!

@theodab can you review it? Thanks!

  I felt that I couldn't make the setting easy to explain as a boolean, so I both renamed it and made it into an enum.  Please let me know what you think.

@joeyparrish Works for me, thank you !


Why deacc?

I see that the parameter deacc is set to True for all languages when tokenize is called from clean_text_by_word and tokenize_by_word. This means that in Nordic languages, characters with umlaut like å, ä, ö are rendered without umlaut as a, a, o; this drastically reduces the quality of the keyword extraction for Swedish which makes heavy use of umlauts and where the umlauted characters count as characters in their own right (For example snö=snow, sno=twist; äga=own, aga=spank).
AFAICS, it's easy to set deacc=False and obtain much nicer results for Swedish; it should probably be done on a per-language basis (I suggest False for danish, finnish, german, norwegian, and swedish).
I'll be happy to file a PR but I'd like to hear if you have any comments?

Hey @emillundhcodemill, thanks for your contributions!
I actually don't recall why the deaccent is always set to true. In our mother tongue (Spanish) few words can be written with or without the accent (solo, for instance). In most cases, though, the accent mark is used consistently to show intonation or to differentiate between same words with different meaning.
I would suggest to default deacc to False in all cases and to make it a parameter in the keywords method. What do you think @fedelopez77?

I totally agree with @fbarrios. Honestly we don't remember why we left that with True.
@emillundhcodemill if you could just set it to False by default, and add it as an optional parameter in the keywords method, we would be more than happy to merge your contribution.
Thanks in advance!

OK, done. https://github.com/summanlp/textrank/pull/22