Example lora_config.yaml not up to date

When updating the config yaml for the addition of full parameter tuning, the lora_layers parameter was not adjusted to num_layers.
https://github.com/ml-explore/mlx-examples/blob/9000e280aeb56c2bcce128001ab157030095687a/llms/mlx_lm/examples/lora_config.yaml#L17
However, if the config is used, both values are later found in adapter_config.json. I assume that only the num_layers value is valid and lora_layers is only copied? I probably have to fine-tune my model again if I want to use the correct number of layers?
Maybe it also makes sense to include a check that only valid values are included in addition to updating the lora_config.yaml?
    "lora_layers": 32,
    "lora_parameters": {
        "keys": [
            "self_attn.q_proj",
            "self_attn.v_proj",
            "self_attn.k_proj",
            "self_attn.o_proj",
            "mlp.gate_proj",
            "mlp.down_proj",
            "mlp.up_proj"
        ],
...
    "model": "meta-llama/Meta-Llama-3.1-8B-Instruct",
    "num_layers": 16,

Are there any other values that I should update in the config? I have seen that use_dora has also been dropped?

Yes lora_layers is ignored. It should be removed from the example config.
We could have a check.. but then again.. it might be useful to let the user include other stuff in the config (some metadata for their own experiments).

Yes lora_layers is ignored. It should be removed from the example config.
We could have a check.. but then again.. it might be useful to let the user include other stuff in the config (some metadata for their own experiments).

Ok. Thank you. Should I create a PR for the example_config.yaml or do you want to do that?
That's a good point. I didn't realize it was intended to include metadata. I always created an extra json file with additional metadata. Maybe a deprecation warning would have helped me to recognize earlier that the parameters had also changed in the yaml.  But that's not meant to be a criticism. I am grateful for your work on MLX LM and think you are doing a great job. I just realized by coincidence that I still had the old lora_layer-parameters in the config.

Thanks for the fix! For the future a deprecation warning is the right call.. we'll be more careful there.


Feature Addition - Graphical / Table view of Swarm

Have a graphical or table based view showing all nodes in a swarm cluster, and the containers running on each node. this should be akin to Swarm Visualiser

I found this project because I was looking for a feature like this.
One other things that I wanted was the ability to see the networks each container is connected to in a visual way.

@paulkling checkout weavescope for that.  It sounds exactly like what you want.  I can't enter links, so you will need to google it.


[Question] Is formio.full.min.js required?

Do I need formio.full.min.js for react-formio? A lot of components seem to work fine without it, so is it the case that react-formio has a built-in npm dependency on that package?

You do not need formio.full.min.js for react-formio. If you are using a module bundler, formio.js will be bundled with it automatically. You may need to either include formio.css or at least the CSS for choices.js and flatpickr or those components will render unstyled and look funny.

Ah okay, sweet. Any chance I could get some links to the <link> tags required for choices.js and flatpickr just in case I need it for one of the components?
Thank you!

I am closing the issue as it was created too long ago and there are no new comments here. I hope it was resolved. If not, please reopen it. Thanks!


types: export RefSymbol

If you decide to use readonly from vue you will face the following issue:

The issue is that RefSymbol is not exported.
Another example using it directly inside setup:

Could you provide a complete code sample? a screenshot where the code is covered in large parts is hard to reproduce. A complete reproduction in issue would be even better.
I ask because I regularly use readonly and haven't experienced this issue.

Could you provide a complete code sample? a screenshot where the code is covered in large parts is hard to reproduce. A complete reproduction in issue would be even better.
I ask because I regularly use readonly and haven't experienced this issue.

If you are trying to export a readonly value from a component, you will get an issue:
setup(){
    const a = readonly(ref(1));

    return {
      a,
    };
  },

I am getting this error because I want to make a composable exporting a readonly value(it 's reactive but should only be modified from the composable side, not in components), and it should bind to the template. @LinusBorg

Full code logic here, useA is a composable.
<script lang="ts">
import { DeepReadonly, defineComponent, readonly, Ref, ref } from "vue";

interface AResult {
  a: DeepReadonly<Ref<number>>;
  updateA: (val: number) => void;
}

const useA = (): AResult => {
  const a = ref(1);

  const updateA = (val: number): void => {
    a.value = val;
  };

  return {
    a: readonly(a),
    updateA,
  };
};

export default defineComponent({
  setup() {
    const { a, updateA } = useA();

    return {
      a,
      updateA,
    };
  },
});
</script>

Hm, this problem only happens with Vetur, not Volar. Does it break on build? Haven't had time to check that out.
Anyway, I would suspect the issue is rather with DeepReadonly, as it breaks the Ref type and turns it into a normal object type:
https://github.com/vuejs/vue-next/blob/fbf2d68bbcde75c95363e1fd17946c720d0f3925/packages/reactivity/src/reactive.ts#L130-L131
If we added a check for like this before the generic object one, this might be fixed without exposing  the internal symbol.
Will try and come up with a fix for that so that we don't have to expose this internal Symbol type.

Hey, sorry for getting back late here, I was not available in the past ~2 weeks.
I've setup a branch in the project using readonly: https://github.com/viniciuskneves/vue-snakke/blob/typescript-readonly/src/services/Snakke.ts#L20
I want to run: npx tsc --emitDeclarationOnly to generate my declarations.
If I run it in the above branch, I get the following:

src/services/Snakke.ts:3:25 - error TS4058: Return type of exported function has or is using name 'RefSymbol' from external module "/Users/viniciuskneves/workspace/own/vue-snakke/node_modules/@vue/reactivity/dist/reactivity" but cannot be named.
3 export default function useSnakke() {
~~~~~~~~~
Found 1 error.

If I remove it, I get the declarations as expected.
By the end it is not just an IDE/plugin issue, I think.
Does it help?

Hey @LinusBorg, did my last message helped somehow?

Hey, thanks for the gentle reminder. Need to get back to this, will look at your branch this week as I have the week off. If that branch helps me to reproduce the error it will definitely help.
Already played with a solution but was not sure how to to add a proper test for it without a way to make it fail before ...


Hi

Hello, dear supporting!
What is this, tell me please

You can hover over that warning icon to see what's going on.
Your wallet is not connected to the backend which prove the exchange rate.

Tell me how to fix this pls

Leave the wallet open for a while it should do that automatically.
If it is not fixed close Wasabi and restart it. By default Wasabi is kept running the background so close it from the tray).

@GGharvestprince I assume this got fixed for you. If it is not feel free to reopen the issue.


How to process large videos?

As far as I can see you're storing the input video as an list of frames and then you're processing them. But if the video is long enough my PC is just running out of memory when storing all the frames in a list obviously. How can I process long videos then? Should i divide them in smaller ones first or it's possible to process the video frame by frame using video.read()?

You are correct.
In predict.py, it stores all the frames in the video as a list and utilizes this list to form the dataset, including estimating the background and generating input sequences. If you prefer not to save any additional data, you could consider using a sliding window and a buffer to process partial video segments instead of utilizing the data loader. Additionally, if the camera viewpoint remains fixed, you can simply use portions of the video to estimate the background.


fixAdd FE metadata-related file checks

Proposed changes
Issue Number: close #xxx

run buildall

run buildall

run buildall

run cloud_p0


Protected Entities - base entity/interface that all protected entities inherit from

There are inconsistencies in filtering for ownership/access across different Entity types in the application, with duplicate logic spread over multiple entities.
A generic solution that applies to all protected would allow for consistent filtering/counts etc based on ownership and visibility

Related issue: https://github.com/Hyperfoil/Horreum/issues/514

Fixed by : https://github.com/Hyperfoil/Horreum/commit/ecdb2ba76597c07af90cfd3bd4d1b3ccf87582c9


[image_picker] Report Permissions Status

Use case
The image_picker plugin handle very well the process of asking the permission to take a photo or choose from gallery but if the user refuse the permission, there no way to know that. Without additional code, the user could think that the app is unresponsive of broken.
It would be nice to know if the user refused a permission to show him some information or redirect to the settings.
I know there are others plugins that can do that but i think every app that use this plugin should handle the case of a refusal, so this should be included in this plugin.
Proposal
A method to know the status of the permission (don't know / approved / refused)
A method to redirect the user to the settings

Even an Exception if it was denied would be fine.

Why flutter image picker doesn't require android permissions for capturing gallery or camera photos. While it requires permissions for iOS part?

Verified this issue using latest image_picker plugin version on Android and iOS and noticed that, after denying permission, the app shows message on screen, as expected.

flutter doctor -v
[✓] Flutter (Channel stable, 1.22.6, on Mac OS X 10.15.4 19E2269 darwin-x64, locale
    en-IN)
    • Flutter version 1.22.6 at /Users/dhs/documents/Fluttersdk/flutter
    • Framework revision 9b2d32b605 (3 weeks ago), 2021-01-22 14:36:39 -0800
    • Engine revision 2f0af37152
    • Dart version 2.10.5

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.0)
    • Android SDK at /Users/dhs/Library/Android/sdk
    • Platform android-30, build-tools 30.0.0
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    • CocoaPods version 1.9.3

[!] Android Studio (version 4.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)

[✓] VS Code (version 1.53.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.18.1

[✓] Connected device (2 available)
    • SM G975F (mobile)          • RZ8M802WY0X                          • android-arm64 •
      Android 10 (API 29)
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729 • ios           •
      com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)

! Doctor found issues in 1 category.

I believe this message is enough to know the status of the permission and indicates what the user need to do.
Closing this as fixed. If anybody disagrees, write in comments and I'll reopen it.


Custom eases not rendered correctly

@bodymovin Attached is a sample AEP with a custom ease on the scale of a triangle and a video of what it looks like After Effects vs Bodymovin. It looks the same as bodymovin in Lottie.
Originally reported here: https://github.com/airbnb/lottie-android/issues/308

Forgot to attach the zip
Ease.zip

@bodymovin this was originally a bug I filed on lottie-android, thinking it was only on Android. I'm also experiencing some errors with layer parenting on iOS that I'll email you about directly.

hi, it should be fixed on the latest build
v4.6.11
you'll need to install the latest extension and reexport the animation

@gpeal @bodymovin Looks good, thanks for the fix guys!


(#3569) Set Cake runner project as the default Startup Project

Closes https://github.com/cake-build/cake/issues/3569

@augustoproiete thanks for getting this updated!


Estimate size of resulting time machine and report to user

CRF28: patp: 3648 x 2763 x 1554 = 15.66 gigapixels.  Size is 8.7 GB for mp4+webm.  4.44 bits / pixel.
brassica: 19704 x 7744 x 2950 = 450 gigapixels.  Size is 173GB for mp4+webm.  3.08 bits / pixel
CRF26:  heinz breathecam 2015-01-03:  8970 x 1733 x 1428 = 22.2 gigapixels.  Size is 23 GB for only mp4.  8.3 bits / pixel.
CRF24: USDA  24764 * 8076 * 168 = 33.6 gigapixels.  Size is 103GB for both.  24.5 bits / pixel
How about:
30:  3.5 bits / pixel
28:  5 bits / pixel
26: 16 bits / pixel
24: 25 bits / pixel


Add an option to hide the average response times

Now, you can add an option to hide the average response time for a site. For example:
sites:
  - name: Google
    url: https://www.google.com
    showAverageResponseType: false

I have read the CLA Document and I hereby sign the CLA

recheck

recheckcla

It seems that the CI is not working. I have accepted the CLA. What can I do?

+up?

This would be very useful.


Implement (normalized) mutual infomation score

http://scikit-learn.org/stable/modules/generated/sklearn.metrics.mutual_info_score.html

@TomAugspurger Can I give this a shot?

Yes, please do! Let me know if you have any trouble getting started. http://ml.dask.org/contributing.html has the contributing instructions.

@kshitij12345 Are you actually working on this?

@LuisMoralesAlonso  actually was a bit busy, was planning to work on it this weekend, will inform if I am unable to

thanks!!!!


Pull from upstream for non-git upstreams

Context: last point from https://github.com/packit/packit-service/issues/1879#issuecomment-1429223256
Currently, in pull_from_upstream we support any Git repository as upstream. The next step, which is the goal of this epic is to support any upstream, not requiring to be a git repository.
Tasks:

[ ] packit/packit#2099
[ ] #2203
[ ] dashboard changes if needed
[ ] marketing: Mastodon, blogpost, demo, etc.

Could you share examples of such (non-git) upstreams? Thanks

I remember there should be e.g. multiple ruby packages like this, but @lachmanfrantisek who I discussed this with should know some concrete examples.

E.g. Pavel Valena wanted this for the Ruby packages, but there are more upstream projects that don't work with GIT. The thing is, that we don't need to do anything with the git -- the source is downloaded from the URL specified in specfile, specfile is taken from the dist-git and the only info taken from upstream might be changelog if they use that way of changelog generation (what is not always wanted/needed).

Hello, thanks for looking into this!
I have several examples:
rubygem-globalid (most likely there's up to several hundred similar packages):
https://src.fedoraproject.org/rpms/rubygem-globalid//blob/rawhide/f/rubygem-globalid.spec
Update PR would look like (this one is created by my automation):
https://src.fedoraproject.org/rpms/rubygem-globalid/pull-request/4
Source is hosted on rubygems.org (downloaded directly or via gem fetch ...):
https://rubygems.org/downloads/globalid-1.2.1.gem
New versions can be determined via anytia (release-monitoring.org), or here:
https://rubygems.org/gems/globalid
In this case, git is needed for additional Source1, which also needs to be created/uploaded (but there are various/numerous approaches to this).
Apart that, the spec file doesn't need anything from upstream git; simple changelog message is enough. (Sometimes, tagged git release even differs from the gem file; plus, the gem files are checksumed & accepted as the source of truth for Ruby community).

Example without the additional source:
https://src.fedoraproject.org/rpms/rubygem-fog-libvirt/pull-request/4
Another example with sample file change:
https://src.fedoraproject.org/rpms/rubygem-addressable/pull-request/4
Additional to sources file, .gitignore is usually updated by fedpkg, but that can be solved by glob (it's  not yet everywhere):
https://src.fedoraproject.org/rpms/rubygem-ethon/pull-request/2#request_diff

packit just need to know the new version , as the https://github.com/fedora-infra/the-new-hotness do .
After that is just run :
package=$(basename $(pwd))
bugs_str="Update $package to ${version} (#$bugid)"
rpmdev-bumpspec -n $version -c "${bug_str}" $package.spec
spectool -g $package.spec (to download the source)

packit just need to know the new version , as the https://github.com/fedora-infra/the-new-hotness do . After that is just run :
package=$(basename $(pwd))
bugs_str="Update $package to ${version} (#$bugid)"
rpmdev-bumpspec -n $version -c "${bug_str}" $package.spec
spectool -g $package.spec (to download the source)

It might be helpful to add the bug on another line like so:
https://src.fedoraproject.org/rpms/rubygem-rails/c/3e4297b98bfbc40fdca58f97fb1d529d28270ae1?branch=rawhide
Bodhi then should auto-close the bug.

For python-selenium, we currently pull from PyPI (not ideal since it does not ship test cases, but the upstream git repo is gargantuan and also contains non-Python code...). The repo takes a few minutes to clone, and worse, for the last 4.20 release upstream monitoring noticed the PyPI release half an hour before the git repo is actually tagged (guessing the pypi release was done by hand).
So being able to track the actual upstream used by the package would be great

@michel-slm thanks for writing down your usecase!


Design a landing page

It should redirect to the login page

Hey can I work on this issue.

Sure.
But I will assign you this issue when DCP will start from 10th July.

ok

@RR-Sahoo any update ...?

@QAZIMAAZARSHAD  actually I have my sem exam going on so can u assign this to others who are interested

can I work on this?

Please review the landing page as there is not much more so this is what I made.
If you want to suggest anything then you can else I make PR if You like my work.
Well I will make mobile filtration section as well but its not in your page therefore I didn't added it

I tried to use the same images that was in your home page but other images can be used in place of that.
Here is the footer Section

@Shivani13121007 your design looks nice but this is the home page.
By landing page, I meant something like  this

After clicking get started user will be redirected to the login/signup page
After signing in user will be redirected to the home page

Ohhh okk i got your point...
I'll complete it by today

How can i get the updated code of index.html file?
As I did git fetch upstream But it shows the same navbar as that was  earlier

See it is saying this branch is 51 commits behind my branch.
You will see the fetch upstream on the right side.
Click on it.
Then clone the updated repo.

Is it fine?

Looks nice.
Make sure this page should be index.html.
Rename index.html to home.html.
Redirect the Get Started button to the login page.
Change the location of links in all pages, since index.html will be home.html.

so when should I display index.html file?
I mean when user clicks on home.html file he will be redirected to home page
so should I Redirect user on index.html page when he clicks on logo?

I'm done with these 4 instructions:
Make sure this page should be index.html.
Rename index.html to home.html.
Redirect the Get Started button to the login page.
Change the location of links in all pages, since index.html will be home.html.

Fine make PR.
Also, share the link to your website so I can review your changes.

https://elated-wozniak-048845.netlify.app/


Better results with Memory Search and Redis?

IAsyncEnumerable searchResults = kernel.Memory.SearchAsync(Constants.MemoryCollectionName, search, limit: 3, minRelevanceScore: 0.8);
appears to be working great.
There are a few things I need to understand about this:

is there a way to get the same results Pythons AgentExecutor can get/Does the LLM have a way to do thought vs observation and pick the best one?
How can you make sure the max tokens used during this search does not go above open AI upon the results? Is there a chain method?
How are people linking this to get OpenAI type results instead of similarity results?

I have ended up creating a custom system for people to decide and configure their own AI in C#


842 UI for Products list on Solutions Page

Please always provide the GitHub issue(s) your PR is for, as well as test URLs where your change can be observed (before and after):
Fix #842
Test URLs:

Before: https://main--danaher-ls-aem--hlxsites.hlx.page/
After: https://842-ui-for-products-list--danaher-ls-aem--hlxsites.hlx.page/us/en/solutions/mabs/cell-line-development/products

@sharanagoudapatil There are no analytics changes in this PR. Please add the analytics.


Impossible to click a line

I believe a two-point path is currently only selectable via the outline. It should probably be hoverable and selectable in the canvas.

Yes, and this is an easy fix!
The Model generates Graphic objects which represent every mark that will be drawn to the screen. The Graphic object is responsible for rendering (via canvas) and hit detection (using canvas isPointInPath).
So to make lines hit detect properly, you'll just need to modify Graphic.Path's hit detection to use canvas's isPointInStroke in addition to isPointInPath,
https://github.com/cdglabs/apparatus/blob/master/src/Graphic/Graphic.coffee#L107
Looking forward to your first PR @joshuahhh :wink:

such excite


Parameters passed to web viewer parsed incorrectly when more than one = is present in a single key-value segment

Take the following URL for example:
https://mozilla.github.io/pdf.js/web/viewer.html?file=https://cors-anywhere.herokuapp.com/https://drive.google.com/uc?id=1kTywHvNtS2I43rvhDZDaCSZw6Poz7MBX#page=1
The URL points to a dummy PDF that is a copy of the one found on w3c's website. Opening it in Google Chrome 85.0.4183.102 (Official Build) (64-bit) results in:

As you can see, the error (400 Bad Request) shown in the console is due to the file URL being truncated at the second =.
The relevant code that parse the key-value pairs is found here:
https://github.com/mozilla/pdf.js/blob/26ae7ba2ad97b50fedfbd714d703c3d0a63c53a3/web/ui_utils.js#L211
Changing it to split on /=(.*)/ resolves the issue.
const param = parts[i].split(/=(.*)/);

Accidentally submitted the issue before it is fully edited. I had closed the issue and now reopening it after editting.


Delay is not a function

xray.delay(from, [to])
is in the docs but is it supported?

I see it's now answered in #63


Search yields 0 results

Search yields 0 results

@teolemon I already noticed that keyword search was not always effective. I found out that keywords in English work better.
To be investigated in KeywordProductQuery.

@teolemon I already noticed that keyword search was not always effective. I found out that keywords in English work better.
To be investigated in KeywordProductQuery.

ah ok. Weird:
Beef lasagne: 3 results
Bolognese lasagne: many results
https://world.openfoodfacts.org/cgi/search.pl?page_size=500&tagtype_0=categories&tag_contains_0=contains&tag_0=Bolognese+lasagne&search_terms&lc=en&fields=%2Cproduct_name%2Cbrands%2Ccode%2Cnutrition_grade_fr%2Cimage_small_url%2Cselected_images%2Cquantity%2Cserving_size%2Cproduct_quantity%2Cnutriments%2Cnutrient_levels%2Cnutriment_energy_unit%2Cadditives_tags%2Cingredients_analysis_tags%2Clabels_tags%2Cenvironment_impact_level_tags%2Ccategories_tags%2Clang%2Cattribute_groups&json=1

ah ok. Weird:
Beef lasagne: 3 results
Bolognese lasagne: many results
https://world.openfoodfacts.org/cgi/search.pl?page_size=500&tagtype_0=categories&tag_contains_0=contains&tag_0=Bolognese+lasagne&search_terms&lc=en&fields=%2Cproduct_name%2Cbrands%2Ccode%2Cnutrition_grade_fr%2Cimage_small_url%2Cselected_images%2Cquantity%2Cserving_size%2Cproduct_quantity%2Cnutriments%2Cnutrient_levels%2Cnutriment_energy_unit%2Cadditives_tags%2Cingredients_analysis_tags%2Clabels_tags%2Cenvironment_impact_level_tags%2Ccategories_tags%2Clang%2Cattribute_groups&json=1

It's more a keyword search than a word search. I couldn't find anything with "nutella" for instance.

It's more a keyword search than a word search. I couldn't find anything with "nutella" for instance.

@teolemon Probably the lc=en is part of the problem.
In KeywordsProductQuery and GroupProductQuery we explicit ask for language: OpenFoodFactsLanguage.ENGLISH
We should put the correct OpenFoodFactsLanguage instead.

@teolemon Probably the lc=en is part of the problem.
In KeywordsProductQuery and GroupProductQuery we explicit ask for language: OpenFoodFactsLanguage.ENGLISH
We should put the correct OpenFoodFactsLanguage instead.

Fixed by #80.

Fixed by #80.


[#121] Update wiki-content for essential, optional and rejected features

#121
What happened 👀
To prevent this repository from becoming a dumping playground, we need to define some ground rules. Originally we were planning to define must-have vs nice-to-have features, but after some discussions, we realized there's a third category when the feature should not be included in the template at all. So I've revised the wording to the following:

Essential
Optional
Rejected

In the future, we aim to include all essential features by default and have flags for the optional features, while rejected features are ignored. For more information, please check the code changes.
Insight 📝
To know more about how we came to this agreement, please check the following threads:

https://nimble-co.slack.com/archives/G41UCTGHL/p1629781993001700
https://nimble-co.slack.com/archives/G41UCTGHL/p1629954695014500
https://nimble-co.slack.com/archives/G41UCTGHL/p1642413515010600
https://nimble-co.slack.com/archives/G41UCTGHL/p1642573191009000

Proof Of Work 📹

Next step, code integration part for:

Add optional flags to the generate script command
Add/remove nice-to-have features to the code

I'll create an issue for it tomorrow 💯

@Wadeewee I'll re-think about what examples to give, thanks for the suggestions 👍

@Tuubz It's a good idea but I'm not sure about must-have sometime it depend on client which is they want us to implement or depend on type of project as well because some project may be don't need API so we no need to use Retrofit, I feel like all of your sample are nice-to-have, I think for must-have it should be generic that not depend on feature something like these kotlin, mocck, kotest, timber What do you think? 🤔

@Wadeewee Originally, I agreed with your statement, but after some discussions I've changed my mind 🙏
It's true that the project depends on the client's requirements, but most of the time, they leave it up to us to decide which technologies to use for a new project, which we believe are best for the team and project 🚀  In terms of the Retrofit example, I still think it's an essential part. Correct me if I'm wrong, but I think it's unlikely that a project will not require an API. 🙇
Anyhow, I've updated the content. Please have another look when available 🤩

@Tuubz It's a good idea but I'm not sure about must-have sometime it depend on client which is they want us to implement or depend on type of project as well because some project may be don't need API so we no need to use Retrofit, I feel like all of your sample are nice-to-have, I think for must-have it should be generic that not depend on feature something like these kotlin, mocck, kotest, timber What do you think? 🤔

@Wadeewee Originally, I agreed with your statement, but after some discussions I've changed my mind 🙏
It's true that the project depends on the client's requirements, but most of the time, they leave it up to us to decide which technologies to use for a new project, which we believe are best for the team and project 🚀 In terms of the Retrofit example, I still think it's an essential part. Correct me if I'm wrong, but I think it's unlikely that a project will not require an API. 🙇
Anyhow, I've updated the content. Please have another look when available 🤩

@doannimble this proposal has a few flaws that you are leaving for further confusion:

Must Have - which cannot be missing and or replaceable.

there is nothing not replaceable forever...so it's either the only option for now (navigation component), or there are 2 ways to do things but 1 must be the default (coroutine over rxjava).

Nice to have - which can be replaceable, but so far, it is our best choice.

Now when you read again, you actually reference  but so far, it is our best choice. to the first choice

Optional - which can be replaced by other ones anytime.

and lastly, this same repetition which can be replaced will create confusion to categorize type with type 2.
Please understand the idea first, having an opinion and cross-check on your understanding before proposing a new idea.

@minhnimble @lydiasama @Wadeewee @sleepylee Updated 🚀

@luongvo @sleepylee @lydiasama @Wadeewee Updated 🤩

@doannimble @sleepylee Any final words? Please do, if any 😁


Online demo broken

Hi, I've tried to go on your website and try some queries, but it looks like it's down right now

Online demo is not official. Please open issue in it's repo.


Docker starting web page

I think the docker version should launch in "Play" mode then when you go in to "feeds" to load add a button there to select edit to edit/create frees to get to that section. Seems like that would make more sense to me. I don't think the info web page make sense when everything is setup for people to play. They have already checked out everything & set it up & now they just want to play.
Just my personal opinion.

The reason is they may want to create a feed with the editor.

This is why I suggested just adding an edit button to the screen where feeds are selected. It makes more sense to find the edit feeds there than the current home page/landing page.

You are free to fork it and do that. Show me how you would make it work. I personally don’t think it would be better. But, that is the great thing, everyone can have an opinion, and with open source you can modify and propose.

Please provide a PR with your suggestion. Thanks!

No activity, closing.


Adds TO client api tests for DS automation

What does this PR (Pull Request) do?

[ ] This PR fixes #REPLACE_ME OR is not related to any Issue

Which Traffic Control components are affected by this PR?

CDN in a Box
Traffic Ops
CI tests

What is the best way to verify this PR?

If this is a bug fix, what versions of Traffic Control are affected?

The following criteria are ALL met by this PR

[x] This PR includes tests OR I have explained why tests are unnecessary
[ ] This PR includes documentation OR I have explained why documentation is unnecessary
[ ] This PR includes an update to CHANGELOG.md OR such an update is not necessary
[x] This PR includes any and all required license headers
[x] This PR DOES NOT FIX A SERIOUS SECURITY VULNERABILITY (see the Apache Software Foundation's security guidelines for details)

Additional Information

I am not sure why I am getting errors when I am adding these tests. It's working fine in local.

Error message indicates that testData.DeliveryServices[0].ProfileName is nil - are you working with the same tc-fixtures as master?

yeah, on master I can see:
            "profileName": null,

So your tests are depending on a specific Delivery Service in the test data having a ProfileName that is safe to dereference, but it just doesn't. It's possible that this is because the fixtures data has changed since the time these tests were written.

yes, that's fine. I will update it. But this error is not related to the profile name null value.
Error Message: Traffic Ops: ERROR: traffic_ops_golang.go:161: 2021-04-14T04:50:58.152866453Z: listen tcp <IP_ADDRESS>:6060: bind: address already in use

Sounds like your port 6060 is already in use and it's trying to start Traffic Ops listening to that port. netstat -nlptu | grep 6060 should be able to find the PID/Process name using it. You might need to use sudo if the process isn't owned by your user

Re-opening pull request to check errors.


feat: CURP WAL Segment

based on: #571
Please briefly answer these questions:

what problem are you trying to solve? (or if there's no problem, what's the motivation for this change?)
Implement CURP WAL

what changes does this pull request make?
This PR implements the WAL segment part.

are there any non-obvious implications of these changes? (does it break compatibility with previous versions, etc)

The segment code will be covered by tests in later PRs


"404: '@apidevtools/swagger-cli@latest' is not in the npm registry" when installing package

I don't know what really went wrong but npm seems unable to find the swagger-cli package in the registry. This is the error log I get when I run npm install -g @apidevtools/swagger-cli
npm ERR! code E404
npm ERR! 404 Not Found - GET https://registry.npmjs.org/@apidevtools%2fswagger-cli - Not found
npm ERR! 404
npm ERR! 404  '@apidevtools/swagger-cli@latest' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/idoko/.npm/_logs/2020-08-06T09_12_34_106Z-debug.log

And it happens both on my dev machine and Github Actions.

could be related to global NPM access issues https://status.npmjs.org/incidents/cksjqc1w11v5

Yeah, this is caused by the NPM problem that @andrejs-sisojevs-accenture pointed out.  It's just temporary. Once NPM resolves the issue, everything should work again.


IE 11 double top bar space

IE11 seems unable to figure that the body is set to display:relative, despite code checking for it explicitly. The Google Translate widget sets the body to display:relative (which is what causes our problem). That feels safe to do, so we'll just mimic and always set position to relative, so that position check block isn't needed.


[Feature Request]: Update README for navigable table of contents

Is there an existing issue for this?

[X] I have searched the existing issues

Feature Description
I would like to enhance the README file by adding a navigable Table of Contents that links to specific sections. This will improve user experience by allowing easy access to different parts of the documentation. Ensure that all relevant headings have corresponding links in the Table of Contents.
Use Case
this would make it easy for navigating various section in README
Benefits
No response
Priority
High
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC contributor
[X] I want to work on this issue
[X] I'm willing to provide further clarification or assistance if needed.

@mansiruhil13 please assign me this task

@mansiruhil13 please assign me this issue


Added missing tests in test_compiler and test_ir_generation

Coverage remained the same at 94.341% when pulling 7ad7602ceba933b4641ae656cd477921dff37b12 on amartyashankha:missing_tests into 51e183a2111af8152ad9b0e8fbad6eed8ff7e2e9 on kensho-technologies:missing_tests.

@jssandh2 I'm not sure I understand your request? Is it the same formatting thing I commented on, or is it something else?

The irony when you add tests, and coverage goes down :)


adding connector for the Python DB 2.0 API

This PR introduces a connector in accordance with the Python DB API 2.0 standards.
In my initial implementation I created a separate repository https://github.com/TileDB-Inc/TileDB-Cloud-PythonDB, but @ihnorton suggested that we should include the connector here instead.
All the code is included in this PR but I am not sure how to package it correctly. Any suggestions are welcome @thetorpedodog

I have applied/responded to all comments above and rebased against main.

This all looks good. Squash and merge at your leisure!


how to pass a custom logger to the fastapi app

First check

[ ] I used the GitHub search to find a similar issue and didn't find it.
[ ] I searched the FastAPI documentation, with the integrated search.
[ ] I already searched in Google "How to X in FastAPI" and didn't find any information.

Description
Great! The performance of fastApi is impressive. I am going to deploy it. And I want to custom a  logger. Can I pass it to the app?

Yep, what @Slyfoxy said.
You can get the logger with:
import logging

logger = logging.getLogger("fastapi")


MODELIX-542 Deduplicate GenerateAntScriptForMps

Depends on https://github.com/modelix/modelix.mps-build-tools/pull/15

:tada: This PR is included in version 3.15.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


CI Tonomy SDK

Done:
When a commit is pushed:

[ ] install eslint with airbnb linter on repository
[ ] linting cheques should be  carried out, If any errors or warnings then fail
[ ] Run any unit tests

Duplicate of https://github.com/Tonomy-Foundation/Tonomy-ID-SDK/issues/3


Lifecycle Fragments

It doesn't seem that the LifecycleFragment actually works in feature modules. I don't think I'm the only one with the issue or if this is the right channel to be posting on. :/

Maybe this issue over on the Google Issuetracker can help you.

I think it should at least be mentioned somewhere in the documentation as our company is looking to use instant apps in tandem with viewmodels

It is documented in the new configurations section of the gradle plugin.

api:
The dependency is available to the module at compile time, and it's also available to the consumer of the module at compile time and runtime. This configuration behaves just like compile (which is now deprecated), and you should typically use this only in library modules. App modules should use implementation, unless you want to expose its API to a separate test module.


PF1: matching things

1 - Need to convert "Feather Token (Tree)" to "Feather Token, Tree" to try and find a match.
2 - Work out how to create potions

Potion - drag spell from compendium onto actor, choose Potion, Scroll, Wand or Spell.
We need to duplicate this functionality.

Potions, Wands and Scrolls for known spells are created using ItemSpellPF.toConsumable


fix "406: Not Acceptable"

Codecov Report

Merging #332 (aadd470) into master (1408d44) will decrease coverage by 0.05%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #332      +/-   ##
==========================================
- Coverage   73.00%   72.95%   -0.06%
==========================================
  Files          44       44
  Lines        9105     9106       +1
==========================================
- Hits         6647     6643       -4
- Misses       1751     1754       +3
- Partials      707      709       +2

Impacted Files
Coverage Δ

server/rest.go
68.57% <100.00%> (+0.02%)
:arrow_up:

master/task_manager.go
88.23% <0.00%> (-3.68%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1408d44...aadd470. Read the comment docs.


samples: canopennode: various updates

Hi, was playing around with the canopennode sample on a custom STM32F107 board with settings on an external SPI flash, bumped into the main stack size limit. Sending out a fix for that and also modernized the GPIO APIs a bit, three patches:

Setting the thread name for the canopen tx_workq
Bump up the sample main stack size
Update the LED apis, get rid of the conditionals in the code

Tested the above on a custom board, dropping the led and switch stuff from the DT works as expected.

Added error handling for gpio functions, also changed few LOG_ERR to log the function return code.


Refine natural keys for VLANGroup and Prefix

[x] Remove overridden natural-key for VLANGroup, allow it to just be name
[x] Investigate feasibility of using prefix instead of network, prefix_length as the composite-key for Prefix

Done by #4027.


Removed links to WDC v1 docs

The WDC v1 docs are no longer available online. Removed links and changed text to indicate that v1 of the WDC is no longer supported.

Go ahead with the merge. There are no conflicts with the dev branch.
Errors are with the CI build and are unrelated to content changes.

Go ahead with the merge. There are no conflicts with the dev branch.
Errors are with the CI build and are unrelated to content changes.


Does not load in Grafana 6.3.3

Here the error message.
Error: Fetch error: 404 Not Found Instantiating
http://localhost:3000/public/plugins/simpod-json-datasource/module.js \
Loading plugins/simpod-json-datasource/module

Try a newer Grafana version.

it is a typo. i will correct at the office later.

corrected, my apologies for the mistake.

@nmingotti how did you install the datasource?

Download from Git, unzip and put the folder into xxx/grafana-6.3.3/data/plugins.

Please follow this https://github.com/simPod/grafana-json-datasource#development-setup and build the plugin

Other plugins usually version built files which we decided is not needed as it only pollutes the repository.

@simPod should still be poissible to use the grafana cli, too?

-] The plugin compiled correctly after downloading a the terrifying number of ~900 node packages.
-]  The plugin works as expected.
thank you for clarification.
bye

@andig the version downloaded from grafana is built. I'll send you PR with release when I do it. Maybe it's time to release new version?
@nmingotti that's the js world for now ;) there are solutions to that but the support is not mature yet.


avcodec/decode.c/avcodec_open2() fails due to missing AVCodecContext extradata

I tried to use in avcodec module MediaCodec h264 decoder:
avcodec_h264dec    h264_mediacodec

That failed in avcodec/decode.c init_decoder() function at
if (avcodec_open2(st->ctx, st->codec, NULL) < 0)

I then traced ffmpeg source code and found that the function call failed because there was no extradata in AVCodecContext parameter st->ctx.
This context is initialized by earlier call:
st->ctx = avcodec_alloc_context3(st->codec);

but that does not add any extradata to the context.
I then found a StackOverflow discussion about populating extracode:
https://stackoverflow.com/questions/35144059/ffmpeg-how-do-i-get-the-extradata-required-to-open-the-alac-codec, but haven't figured out, how to do that in avcodec module.
Any help is appreciated, since the code as it currently stands, does not work.
Here is another reference about the same topic: https://trac.ffmpeg.org/ticket/5838.

I got this reply from the author of ffmpeg h264_mediacodec decoder (Matthieu Bouron), when I asked about the extradata:

Extradata are mandatory for the h264_mediacodec decoder because we want
MediaCodec to fail during init (if the underlying codec does not support
the stream) so the user gets a chance to use a software fallback without
having to wait for avcodec_receive_frame() to fail.

the extradata should contain the Sequence Parameter Set (SPS)
and Picture Parameter Set (PPS). what you need to do is:

save the incoming SPS
save the incoming PPS
when both SPS and PPS have arrived, write them to extradata

The SPS/PPS needs to have the NAL escape sequence in front (0x00000001):
0x00000001 <SPS> 0x00000001 <PPS>

OK thanks.  I'll try later to figure out how I can get access to SPS and PPS.  I don't currently use h264_mediacodec decoder, so this is not an urgent issue. I'll add a TODO comment about this to avcodec module.


flink sql用kafka作为源建表多分区问题

CREATE TABLE topic_products (
-- schema is totally the same to the MySQL "products" table
id BIGINT,
name STRING,
description STRING,
weight DECIMAL(10, 2)
) WITH (
'connector' = 'kafka',
'topic' = 'products_binlog',
'properties.bootstrap.servers' = 'localhost:9092',
'properties.group.id' = 'testGroup',
'format' = 'maxwell-json'
)
kafka中products_binlog这个topic是maxwell同步到kafka写入的，但是如果这个topic有多个分区，
如插入两条数据，产生2条binlog到products_binlog这个topic的 分区1中，
之后又删除了这两条数据，产生的binlog到roducts_binlog这个topic的 分区0中，
flink sql读取kafka数据的时候，先读取分区0中的数据，然后才读取分区1中的数据，
这样会不会flink sql查出来的结果是有 2 条数据，但实际表中已经没有数据了

We should keep the record whose primary key is same shouble be in the same partition.

We should keep the record whose primary key is same shouble be in the same partition.

OK, I know. Another question, if there are binlogs of multi tables in one topic, what if defining a table like this, would this occur some problem.
for example, there is another table with 2 columns besides table topic_products in topic products_binlog

Do you mean the schema of the tables are totally different? Flink SQL requires the source/sink has explicit schema... I think you should send the binlog of the different table to the different topic or partition. Flink Kafka connector also supports to read data from the specific partition[1].
[1] https://ci.apache.org/projects/flink/flink-docs-master/docs/connectors/table/kafka/#scan-startup-specific-offsets

By the way, we should discuss about CDC here. If you have any questions about the usage of the Flink Kafka connector, you should send your questions to the Flink mail list[1].
[1] https://flink.apache.org/community.html#mailing-lists


[AVM Question/Feedback]: Split Location Variables for AVD Resources

Check for previous/existing GitHub issues

[X] I have checked for previous/existing GitHub issues

Description
Issue Proposal: Split Location Variables for AVD Resources
In the current AVM pattern, a single location variable is used for all AVD resources. However, certain AVD components, such as the Workspace and Application Group, necessitate deployment in different regions due to regional support constraints. This constraint is problematic when deploying AVD in regions like Norway East, where AVD Workspaces are not supported. Consequently, I am unable to deploy the Workspace and Application Group in a supported region (e.g., West Europe) while keeping other resources in the desired region.
Current Behavior

The module requires a single location variable.
All resources, including Workspace and Application Group, are deployed in the specified location.
Attempting to deploy unsupported resources in certain regions results in deployment errors.

Desired Behavior

Ability to specify separate location variables for different resources within the module.
Enhanced flexibility to deploy resources in regions that support them without modifying the module code

Hi @PetterHL, Thank you for raising the issue! You raise a good point. I will work on an update. Thank you again!


[eval] fix cv.yaml

The unit test failure is attributed to the issue documented at https://github.com/microsoft/DeepSpeed/issues/4984


Send incoming message to clients not working?

Hi,
Before I explain my problem, let me explain my setup / goal.

A server (in my example I'm just using Postman) sends a response to my website, for example example.com.
The website example.com catches that response ($_POST-variables).
example.com than connects to the already running "Wamp2 Thruway" server and sends the $_POST-variables.
The server then notifies his clients by using $pull->on('message', ...
The clients have an open websocket to the server with AutobahnJS and they receive the message.
The clients who use ReactJS, update their DOM with the new info.

Ok, so here comes the problem:
The website example.com catches that incoming request and tries to send it to the already running  "Wamp2 Thruway" server. But this somehow fails...
<?php

// This is the website example.com

try {
    $context = new \ZMQContext();
    $socket  = $context->getSocket(\ZMQ::SOCKET_PUSH, 'thisIsMyPusher');
    $socket->connect('tcp://<IP_ADDRESS>:5555');
    $socket->send(json_encode($_POST));
    sleep(3);
}
catch (\ZMQSocketException $e) {
    echo "An error occured\n";
    echo "{$e->getMessage()}\n";
}

So, you see I just use send(), but on the server-side nothing happens.
<?php

use React\EventLoop\Factory;
use React\ZMQ\Context;
use Thruway\Logging\Logger;
use Thruway\Peer\Client;
use Thruway\Peer\Router;
use Thruway\Transport\RatchetTransportProvider;

require_once(realpath(__DIR__ . '/../vendor/autoload.php'));

$loop   = Factory::create();
$pusher = new Client('myCallboard', $loop);

$pusher->on('open', function ($session) use ($loop) {
    $context = new Context($loop);
    $pull    = $context->getSocket(\ZMQ::SOCKET_PULL);
    $pull->bind('tcp://<IP_ADDRESS>:5555');

    // Handle incoming messages
    $pull->on('message', function ($entry) use ($session) {

        Logger::info('Incoming message...');
        echo $entry;
        Logger::info($entry);

        // NOTHING GETS LOGGED, PRINTED, ECHOED, WHATSOEVER...

        $entryData = json_decode($entry, true);
        if (isset($entryData['category'])) {
            $session->publish($entryData['category'], [$entryData]);
            // ALSO ON THE SERVER SIDE NOTHING GETS RECEIVED
        }
    });
});

$router = new Router($loop);
$router->addInternalClient($pusher);
$router->addTransportProvider(new RatchetTransportProvider('<IP_ADDRESS>', 59123));
$router->start();

Am I doing something wrong, can it be that port 5555 needs to be opened, although its all "local", is it something else?
I hope you guys can help me out.
Thanks in advance!
Regards,
TK

I don't have zmq in my current setup to help test what your issue is, but you may want to take a look at WampPost which will let you publish or call into the router from your synchronous PHP code like this:
        $body = [
            "topic" => 'my.topic',
            "args" => ['arg1', 'arg2']
        ];
        $context = stream_context_create([
            'http' => [
                'method' => "POST",
                'header' => "Content-type: application/json\r\n",
                'content' => json_encode($body)
            ]
        ]);

        file_get_contents('http://<IP_ADDRESS>:8181/', null, $context);

@mbonneau I understand what you are saying and if it works, it's a nice workaround, but I prefer using ZeroMQ.
The current setup I'm using is just as subscribed by @davidwdan here....so don't understand why this is not working :(

Ok, so your gonna laugh real hard now. For some reason the the extension=zmq.so was not loaded....just checked with phpinfo(). Eventually needed to use the whole path, like so:
extension=/usr/lib64/php/modules/zmq.so.
Anyways, your reaction is much appreciated!


Cloud hosting vendor

Ideally this cloud vendor could provide infrastructure needed for cTAKES (Java?) as well as Node.js
Ideally would support easy Meteor deployment

Also we need MongoDB and ElasticSearch.
So the full list of things we need to host:
Node.js + Meteor
MongoDB
ElasticSearch
cTAKES


remove warning filter in pytest plugin

Description
This PR removes the warnings filter in the asdf pytest plugin that was converting (most) warnings to errors:
https://github.com/asdf-format/asdf/blob/fcff366690d43eb4c3513fe1a677e45259f4ba2c/pytest_asdf/plugin.py#L242-L245
This filter is preventing all other warning filtering from having any effect. This is currently making it difficult to make the downstream weldx job pass as pandas is triggering a warning during several of the schema tests. Ideally we could ignore this warning (as it's a deprecation warning that weldx will need to address) by adding a filter to the weldx tox environment. However this is not possible because of the above warnings filter. Instead we would have to add the pandas warning to the pytest plugin warnings filter. With this PR we can add the warning filter to the tox.ini and it will have an effect.
As this PR removes the warnings filter internal to the asdf pytest plugin this does mean that any library that relied on this to turn warnings into errors during schema tests will need to add their own warnings filter (if they don't already have one).
Checklist:

[ ] pre-commit checks ran successfully
[ ] tests ran successfully
[ ] for a public change, a changelog entry was added
[ ] for a public change, documentation was updated
[ ] for any new features, unit tests were added

Thanks for the review!


⚠ Bofrost.be has degraded performance

In e5e3e7f, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1097 ms

Resolved: Bofrost.be performance has improved in e237dfd.


kde5.kile: init at git 2016-02-12

This adds a kde5/frameworks version of kile.

@ttuegel could you review?
In Kile there is a button with which you can view the pdf in Okular. This does not work because okularpart is not available. I tried adding okular to buildInputs, but still, cmake complained it couldn't find okular.

I tried adding okular to buildInputs, but still, cmake complained it couldn't find okular.

It's probably looking for the Qt 5 version of Okular. There hasn't been a public release of that version yet, so the one in Nixpkgs is still on Qt 4.

It indeed requested version 5. That's clear then, thanks.


Test/gh pages

Testing creating a version file for the gh-pages branch based on the available builds.

I think we can cut this down a bit, then we could make it a script in https://github.com/dls-controls/sphinx_rtd_theme_github_versions

How should this be cut down and how does the script end up being called from the sphinx repo?

Code moved to https://github.com/dls-controls/sphinx_rtd_theme_github_versions


Polynomial linear regression

Both issue for main branch

@kpratik1929 if you merge several issues in one PR the leaderboard will count only one PR merged and will point you accordingly. If you can then please open three different PRs for three different issues. This will help you score more points. Just a tip from my side.

I have done the changes .you can check it now.

@kpratik1929 you've already added the code of this PR in your previous PR. Therefere we're having a merge conflict but becawe you've resolved 2 issues, I'll be closing this PR and your points will be awarded to you based of the issues you;ve closed. Job well done.


Reuse read buffers on Windows

The change adds a pool of fixed-size grpc_slices and reuses them for reading on Windows. Also uses the GRPC_ARG_TCP_READ_CHUNK_SIZE channel arg for determining the slice size, instead of using hard-coded 8KB.

CC @jtattermusch

****************************************************************

libgrpc.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

[trickle] No significant performance differences

[microbenchmarks] No significant performance differences

@sreecha, can you review?

****************************************************************

libgrpc.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

[microbenchmarks] No significant performance differences

Can you share performance data? Just filed #15125 for standardized Windows perf tooling

I don't have detailed data at the moment, but it basically avoids a contention on heap allocation, especially for relatively larger payloads (100kb+) and higher QPS.

ping @sreecha

****************************************************************

libgrpc.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

[trickle] No significant performance differences

[microbenchmarks] No significant performance differences

I'm no longer actively working on related projects, so won't be able to finish this.


Github action for generating docs using nbsphinx

Via a Jupyter discourse thread on using Github actions to automatically build MyBinder images from pushes to a Github repo, I wondered if there is an example nbsphinx Github action defined anywhere?
Eg I note this https://github.com/ammaraskar/sphinx-action

I have never used Github Actions, and I don't know whether anyone has used it with nbsphinx.
Did you try if the above-mentioned Sphinx action works?
I guess it should work fine if you can specify the nbsphinx requirement (which I guess you can do with a requirements.txt file).
I guess the "fancy in-line warnings" will not work because nbsphinx doesn't track line numbers in .ipynb files.

See also #361.

OK, I quickly tried it, and I found two problems:

A Makefile is needed (see https://github.com/ammaraskar/sphinx-action/issues/2)
I don't know how to install pandoc

No need to use Sphinx-action, you can just run the commands as you normally would. I got a first example working on my branch for the PR.
Pandoc and librsvg can be installed with conda, for osx and Linux. If it wasn't for librsvg, it would even work on windows.
With tectonic, you can even compile latex cross platform.

Cool, I'm looking forward to this PR!
As far as I'm concerned, you don't have to support Python 2 in your efforts, I was planning to drop Python 2 support in early 2020 anyway.

If it wasn't for librsvg, it would even work on windows.

Shouldn't it work on Windows?
At least there's a win32 directory: https://gitlab.gnome.org/GNOME/librsvg/tree/master/win32
I guess the problem is that Windows isn't supported on the conda-forge package, right?

With tectonic, you can even compile latex cross platform.

Cool, I didn't know this project!

@psychemedia if you just want to rebuild your binder image, when i.e. master is updated you can use this action.

@s-weigand Thanks for that; I'm wary of rebuilding too often on MyBinder - I often make piecemeal pushes / do edits on single files via Github UI, and some of my Binder images force quite lengthy builds!

In the meantime, I've used Github Actions in a different project, see https://github.com/AudioSceneDescriptionFormat/splines/blob/master/.github/workflows/docs.yml.
This uses https://github.com/dante-ev/latex-action for building the PDF. It uses quite a big image, which needs nearly 3 minutes to load. Maybe using tectonic, as @s-weigand suggested above (https://github.com/spatialaudio/nbsphinx/issues/358#issuecomment-566205536), is faster, but I didn't try it (yet).


Motor with enable control

Motors on the new MotoZero board require 3 pins to control them. Forward pin, backward pin and an enable pin. They can be controlled with GPIO Zero by using Motor for the forward and backward pins, and OutputDevice for the enable pin like so:
>>> from gpiozero import Motor, OutputDevice
>>> motor = Motor(24, 27)
>>> motor_enable = OutputDevice(5)
>>> motor.forward()  # nothing happens
>>> motor_enable.on()  # the motor goes forward
>>> motor_enable.off()  # the motor stops
>>> motor.backward()  # nothing happens
>>> motor_enable.on()  # the motor goes backward
>>> motor.value = 0.5  # the motor goes forward at half speed
>>> motor.stop()  # the motor stops

My instinct is to want to remove the enabling step, and just let Motor turn on the enable pin on init, and never change its state, allowing forward(), backward(), stop() and value to be used to control the state of the motor.
However, Keith has vocalised his views that the enable pin is particularly useful:

Nice benefit of using enable pin is that it can be PWMed to control the speed, independent of dir.

and

Enable pin 0 = motor off, Enable pin 1 = Motor full speed. Enable pin PWM 0.5 = Motor 1/2 speed

But I don't think this makes it any easier than gpiozero already does:
>>> motor = Motor(17, 18)
>>> motor.forward()
>>> motor.value = 0.5
>>> motor.backward()
>>> motor.value = -0.5
>>> motor.reverse()

I propose we add an optional third pin to Motor and set it high on init. This will allow it to easily fit into Motor and Robot.
Any thoughts?

Yes for compatibility adding a third option pin makes perfect sense.  My experience was a while ago with RPi.GPIO when things were not as easy as they are now with gpiozero.
However it would be good to be able to set the enable pin low to enable debug mode, i.e. set enable low and allow code to run without having to detach power to motors.

It's much easier to do PWM in gpiozero than in RPi.GPIO. So yeah.

From a bit of internet searching, it seems like you can control the L293D by either treating the IN1 and IN2 pins as digital, and then PWM the speed on ENABLE1; or you can treat the ENABLE1 pin as digital and then PWM on the IN1 and IN2 pins.
For microcontrollers with limited PWM pins it's obviously better to only PWM on ENABLE1, but I couldn't find any definitive references saying whether one approach was better than the other or not. So perhaps either approach is equally valid?
(In contrast, the datasheet for the L298N explicitly recommends only PWMing the ENABLE pin)

Ah I see, the (Digital_forwards, Digital_backwards, PWM_speed) makes sense now. In which case, I'd rather stick with one implementation than two. Either we opt for #367, and PWM is done on the forward or backward pin, or we opt for having a different motor class for these types of motors, which are controlled via the enable pin.
I prefer the former solution, but if there's enough support for doing it the latter way, we might have to.

From the perspective of an L293 or equivalent,  I don't think it matters how you PWM it, its just a case implementation.  Since gpiozero does the implementation we do not need to care any more, so the solution in #367 works well.
I was not aware the L298 should be PWMed on the enable pin.  Our PiWars 2014 entry used a L298 PWMed on the drive pins, it seemed to work well.  After reviewing the data sheet today the only reason I can see for having to PWM on the enable pin is because of the 'Fast Stop' feature
For 'Fast Stop' to be engaged, enable pin must be high and and the drive pins must be equal.  Therefore if the drive pins are being PWMed, each cycle of the PWM the 'Fast Stop" feature will kick in.  If PWM is on the enable pin the motors will 'free spin' between cycles.
In critical applications this may be important, but in terms of hobby electronics, I'm not sure how important it is.  It might reduce the life cycle of the motor driver or the motor. L298 data sheet attached, refer to page 6 for 'Fast Stop' feature.
en.CD00000240.pdf

@waveform80 what do you think?
We also need to consider the four motor robot. Is that a trivial extension?

Ahh, I was looking at a different datasheet previously - see https://github.com/RPi-Distro/python-gpiozero/issues/300#issuecomment-215999378
Although now I read it again, I see it refers to both L293 and L298 in the same category, so it seems I was wrong to assume they needed to be treated differently. *shrug*


Xamarin.iOS Build Task Doesn't work

Hi
The Xamarin.iOS task always returns an error: "Return code: 1"
I have done the following to try out the build task.

Created a VSO project, and uploaded the code for the TaskyiOS_Starter sample app from Xamarin.
Set up a build with the Xamarin.iOS as the first build step, and configured it to build for the simulator
Installed Xamarin Studio on the Mac, and logged in with a valid trial account
Installed the cross platform build agent on a mac, and connected it to VSO (I have this running in interactive mode)

When I queue up a new build, I can see it hit my agent, but it returns an error, and the log details the following:
-07-15T00:18:50.059Z: [command]/usr/bin/nuget restore /Users/Gary.Jackson/Projects/myagent/_work/build/0fbee2c4e562ab4b6d6b2db37f92b44ffe234542132e10f0e14be3d9fb4b37e0/repo/TaskyiOS_Starter.sln

2015-07-15T00:18:50.359Z: [command]'/Applications/Xamarin Studio.app/Contents/MacOS/mdtool' --verbose build --configuration:'Release|iPhoneSimulator' /Users/Gary.Jackson/Projects/myagent/_work/build/0fbee2c4e562ab4b6d6b2db37f92b44ffe234542132e10f0e14be3d9fb4b37e0/repo/TaskyiOS_Starter.sln

##[Error] 2015-07-15T00:18:50.919Z: Return code: 1

Removing the following lines from the constructor of the local "toolrunner.js" allows the task to run, so I'm not sure if this is a task bug, or possibly something with the toolrunner?

if (toolPath.indexOf(' ') > 0) {
toolPath = '\'' + toolPath + '\'';
}

Yang?
From: Garyljackson<EMAIL_ADDRESS>Sent: Sunday, July 26, 2015 6:25 PM
To: Microsoft/vso-agent-tasks<EMAIL_ADDRESS>Subject: [vso-agent-tasks] Xamarin.iOS Build Task Doesn't work (#346)
Hi
The Xamarin.iOS task always returns an error: "Return code: 1"
I have done the following to try out the build task.

Created a VSO project, and uploaded the code for the TaskyiOS_Starter sample app from Xamarin.
Set up a build with the Xamarin.iOS as the first build step, and configured it to build for the simulator
Installed Xamarin Studio on the Mac, and logged in with a valid trial account
Installed the cross platform build agent on a mac, and connected it to VSO (I have this running in interactive mode)

When I queue up a new build, I can see it hit my agent, but it returns an error, and the log details the following:
-07-15T00:18:50.059Z: [command]/usr/bin/nuget restore /Users/Gary.Jackson/Projects/myagent/_work/build/0fbee2c4e562ab4b6d6b2db37f92b44ffe234542132e10f0e14be3d9fb4b37e0/repo/TaskyiOS_Starter.sln
2015-07-15T00:18:50.359Z: [command]'/Applications/Xamarin Studio.app/Contents/MacOS/mdtool' --verbose build --configuration:'Release|iPhoneSimulator' /Users/Gary.Jackson/Projects/myagent/_work/build/0fbee2c4e562ab4b6d6b2db37f92b44ffe234542132e10f0e14be3d9fb4b37e0/repo/TaskyiOS_Starter.sln
##[Error] 2015-07-15T00:18:50.919Z: Return code: 1
Removing the following lines from the constructor of the local "toolrunner.js" allows the task to run, so I'm not sure if this is a task bug, or possibly something with the toolrunner?
if (toolPath.indexOf(' ') > 0) {
toolPath = ''' + toolPath + ''';
}
—
Reply to this email directly or view it on GitHubhttps://github.com/Microsoft/vso-agent-tasks/issues/346.

I'll investigate on the toolrunner line - that seems like an odd thing to do (if working with path module shouldn't need to manually append paths).  I'll find out the background on that block of code.  Yang, chat with me.

hmmm, looked again and it's just quoting ... we're investigating.  Thanks for finding the issue!

I think I know the issue ... toolrunner calls child.spawn which takes tool path arg and that api should handle.  No need to quote.  But we used to call child.exec (which had all the limits with output etc...) which constructed one long line so we needed that block.
I'll push a fix to vso-task-lib today.

Awesome, thank you, it was a bit of a head scratcher for me as well, that line seemed to make sense..

https://github.com/Microsoft/vso-task-lib/commit/1bf38c685982a121062e83e9ce70d15ba2f411e5

From root of exising agent, npm update vso-task-lib
Or deploy new agent

Great, thanks Bryan.

Agent 0.2.23 and above pull the fix from vso-task-lib 0.4.4 and above


Mqtt sensor not compatible?

Can someone please confirm  that the list-card will not be able to use a list from an mqtt sensor?
https://community.home-assistant.io/t/please-help-me-create-a-traffic-accident-mqtt-sensor-in-node-red/101134/10?

The card reads the entity's attributes as a dict and treats each key/value pair as an item. If you've got your items as a list of dicts, you can't set it directly on the entity attributes, and the card doesn't allow defining an attribute which contains the list of items.
Additionally, there's no way of using templates to convert lists to dicts, but since you're using node-red, you might have more options there.

@DarkFox has the right idea. Going to close this as not really an issue. Feel free to submit a PR for extended use cases

Just for the record, in case someone with the same issue stumbles over this thread, here's how I solved it: https://github.com/DarkFox/ha-config/blob/master/packages/integrations/weather.yaml#L123-L128
xkcd.com/979

Just for the record, in case someone with the same issue stumbles over this thread, here's how I solved it: https://github.com/DarkFox/ha-config/blob/687791a/packages/integrations/weather.yaml#L123-L128

Please forgive my ignorance, but can elaborate a little as I'm not sure I understand.  You solved the issue that I was having by replacing ' with " in some kind of warning string when publishing to an mqtt sensor?

It's because HA's string conversion outputs it with single quotes, which isn't valid JSON, so it won't get parsed by the mqtt sensor.


PVA: Valid adaptive card json prevents publishing to PVA

Repro

Create a non-PVA bot with adaptive card, test that it works.
Create a PVA bot, switch to advanced authoring with composer
Reproduce the adaptive card in PVA bot
Publish

Error
"severity": "Error",
"source": "MoodCheckerDialog.en-us.lg",
"message": "Error occurred when parsing expression 'json({\r\n \"$schema\": \"http://adaptivecards.io/schemas/adaptive-card.json\",\r\n \"version\": \"1.2\",\r\n \"type\": \"AdaptiveCard\",\r\n \"body\": [\r\n {\r\n \"type\": \"Container\",\r\n \"items\": [\r\n {\r\n \"type\": \"ColumnSet\",\r\n \"columns\": [\r\n {\r\n \"type\": \"Column\",\r\n \"width\": \"auto\",\r\n \"items\": [\r\n {\r\n \"type\": \"Image\",\r\n \"url\": \"https://metatour.azurewebsites.net/img/happy.png\",\r\n \"size\": \"small\"\r\n '. missing '}' at '<EOF>'"
}
---

Sample JSON
# SendActivity_kUwtny()
[Activity
    Attachments = ${SendActivity_kUwtny_attachment_fpA8bl()}
]

# SendActivity_kUwtny_attachment_fpA8bl()
> To learn more Adaptive Cards format, read the documentation at
> https://docs.microsoft.com/en-us/adaptive-cards/getting-started/bots
- \```${json({
  "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
  "version": "1.2",
  "type": "AdaptiveCard",
  "body": [
    {
        "type": "Container",
        "items": [
          {
            "type": "ColumnSet",
            "columns": [
              {
                "type": "Column",
                "width": "auto",
                "items": [
                  {
                    "type": "Image",
                    "url": "https://metatour.azurewebsites.net/img/sad.png",
                    "size": "small"
                  }
                ]
              },
              {
                "type": "Column",
                "width": "stretch",
                "items": [
                  {
                    "type": "TextBlock",
                    "text": "I'm sorry to hear that. Feel better soon.",
                    "weight": "bolder",
                    "wrap": true,
                    "BlockElementHeight" : "stretch"
                  }
                ]
              }
            ]
          }
        ]
    }
  ]
})}\```
(backslash inserted for rendering in issue comment)

And another sample json
# SendActivity_3su1l3()
[Activity
    Attachments = ${SendActivity_3su1l3_attachment_jGnBkM()}
]

# SendActivity_3su1l3_attachment_jGnBkM()
> To learn more Adaptive Cards format, read the documentation at
> https://docs.microsoft.com/en-us/adaptive-cards/getting-started/bots
- \```${json({
  "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
  "version": "1.2",
  "type": "AdaptiveCard",
  "body": [
  {
    "type": "Container",
    "items": [
      {
        "type": "Image",
        "url": "https://metatour.azurewebsites.net/img/happysad.png",
        "size": "large",
        "horizontalAlignment" : "center"
      },
      {
        "type": "TextBlock",
        "text": "**Mood Checker**",
        "weight": "bolder",
        "size": "large",
        "isSubtle": false,
        "horizontalAlignment" : "center"
      },
      {
        "type": "TextBlock",
        "text": "Demonstrate simple intent detection",
        "isSubtle": true,
        "horizontalAlignment" : "center"
      }
    ]
  }
  ]
})}\```
(backslash inserted for rendering in bug)

I faced this issue (again) today and logged https://github.com/microsoft/BotFramework-Composer/issues/7450.
Our workaround for now is to move the json() function call to the Activity block, however this is a manual action and should not be required. It seems that the PVA back-end has an error in the validation..
[Activity
    Attachments = $json({SendActivity_eBobKx_attachment_VNSg6J())}
]

Shiproom - lets spike the adaptive card template and see where we land

Shiproom - approved.  please merge today.

My bot is still not able to publish the json.

@scheyal you would need to recreate the response template, which would come with a new format that PVA supports, not reuse the existing one that did not work. Can you please confirm that

I've been able to reproduce this, and the explanation from PVA is that they need to take a bump to a BF lib in order to support the syntax in the updated template that came in with https://github.com/microsoft/BotFramework-Composer/pull/7593
We'll need to deploy a workaround though, as their release timeline for this change is mid June

Shiproom - core P0 scenario - approved


Title input overflow

Currently the post title is a text input, which means it overflows horizontally. Ideally this should just behave like normal text and break to a new line.

I've got a working and reliable solution, the caveat is that the title is now part of the post content and not a separate field.
I've added helper functions to extract and replace the content text of the title so that Ghost can use it for structured data, however right now the title will appear within the content block in templates.
We do have a few options:

We can modify the title helper, excerpt helper, and content helper so that they pull the right content out of the block of rendered HTML or AMP.
We can fork the mobiledoc HTML renderer so that it doesn't draw the first H1.
We can leave the title in the content block and get rid of the title helper within the theme API.

Is there somewhere an explanation of what the desired UX for the title is and why this is so hard to achieve?
Both this issue and the convo in Slack issue jumps from needing a line wrap in the title field to suddenly having to implement really awkward workarounds all over the place, and I'm not able to follow the reasoning.

Agreed, the normal solution for the UI that @JohnONolan has described is to change the text input to a textarea and have it auto-expand to fit the content - is there some other reason we need to move the title into the mobiledoc editor?

Visually it's part of the editor, because of that a user expects:

To be able to press the down arrow and move from the title into the main body.
To be able to press the up arrow from the body and to go back to the title.
To be able to press enter within the title and have it split into the body.
To be able to press backspace within the body and have that block join with the title.
To be able to style the title.
To be able to undo and redo the title as if it's the body.

If the title and body were visually distinct and it was clear that they weren't the same interface then these wouldn't be issues. But because they're combined and people are used to the Medium editor I think they need to be integrated. Because they're combined visually users will expect a integrated experience.

I also have a reliable and intuitive solution for mobiledoc but it will require a bit of a rework of the admin UI and template helpers as discussed above.

Cory and Mathew from mobiledoc suggested another solution to this, and that is embedding different mobiledoc containers within one mobiledoc document, the first for the title, the second for the content.
This solves the issue with having seamless cursor movement between the title and body and has them as independent entities but has a few trade-offs.

You cannot select between the title and the body.
There is not one undo queue.
Pressing enter in the title doesn't split it and generate a new paragraph in the body.

IMHO having the title inside the body is still the best solution from an editing experience.

Ok so, I understand the reason why it's fiddly to merge 2 UI components seamlessly, however, I really feel like I'm missing something when it comes to this part:

I've added helper functions to extract and replace the content text of the title so that Ghost can use it for structured data, however right now the title will appear within the content block in templates.

I don't understand why, if we have to process the data on save to pull the title out into the post.title field, why we can't at the same point also not add the title to the post.html field?

It should be trivial to remove the title from the body, but it has to happen in one of three ways IMHO.

We have a post render step after the mobiledoc is generated to remove the title from the HTML/AMP/plaintext (or whatever), or
We fork each of the necessary mobiledoc renderers so that we can remove the title as it's rendered, or
We modify the mobiledoc on save to pull out the title and load to replace it.

None of these feel like great solutions, although 3 is probably best. But. instead of working with the editor though (which relies on the cursor range object in the DOM), we'd be better placed if we put together a library for manipulating mobiledoc documents outside of a DOM situation. This could be very useful for other things so it's worth investigating regardless.
To be clearer I wasn't talking about pulling the content out of the mobiledoc on save but rather just getting the plain text version of the first block element to use for our metadata rather than anything more invasive to the document.

Can you please point me at the piece of code which takes the existing mobiledoc document and converts it to HTML to store in the database?

https://github.com/TryGhost/Ghost/blob/2d19ae2c6c403438c714f880ff56bd7555cd8583/core/server/models/post.js#L207
converter is an instance of the mobiledoc HTML renderer which has been setup with our cards, atoms, and various other options.
As you can see it's reasonably straight forwards ATM and works almost identically to how showdown works.

So... as I see, the existing solution goes from JSON -> HTML string. We could regex parse this before storing it, given that we know the structure, but regex parsing of HTML is always GROSS.
However, it seems to me this method of getting from Mobiledoc -> HTML is deprecated.
The new method documented here involves a middle step where we have access to a SimpleDOM object. If we update to use this new method, we have a point at which we can manipulate the DOM.
This would allow us to get rid of the H1 element easily?
Also, we could get rid of that awkward wrapping

, if there isn't another way to configure mobiledoc to not do that?!

Oh nice, they've obviously deprecated it since we wrote the front-end code. using simpledom should make life easier.
The question then is do we want to have the title be formatted?

It isn't right now. I don't know if there's any plan or desire to add that @JohnONolan ?

If it's any consolation it's going to be easier to keep the formatting in mobiledoc than not (we'd have to disable the toolbar and formatting hotkeys if the title is selected, and it's going to be even more of a pain if the user selects the title and content and wants to format it).

sorry about being late to this! I have some definite simplifications in requirements here which might help?

To be able to press enter within the title and have it split into the body.
To be able to press backspace within the body and have that block join with the title.
To be able to style the title.
To be able to undo and redo the title as if it's the body.

Definitely not needed. These are edge cases rather than use-cases!

Thanks John,
How about selecting between the title and body?
Anyway, with the above requirements it would appear that a nested mobiledoc solution might be the best option - I have no idea how to implement that though so will need to do a bit of research.
If we go with a textarea for the title then navigating between elements with the up and down keys is going to be challenging.
I'm not convinced that these are edge cases though but rather expected behavior.

Selecting between the title + body would be nice yeah, to copy/paste a whole post out.

I'm not convinced that these are edge cases though but rather expected behavior.

But when you apply the logic of "hey that doesn't work" to each of those points above, the obvious answer is "well don't do that then" -- it's really not a thing that people NEED to be able to do. It's not a valid feature or thing that is required.
 And why should that be a case which is planned for? If anything I would expect pressing enter in the middle of a title to create a line-break within the title itself - which is what happens in most editors

The only way to be able to select the text in both the title and body is to have everything in one mobiledoc document. 😞

I don't think that one is a dealbreaker though. It would be nice - yes - but if that's the ONLY reason to combine the fields, then it's not a good enough reason?

So the only requirement is to be able to navigate with the up and down keys between the body and the title or is that also not a requirement?

The measurement of what's a requirement, what's not, and what's nice to have, is based pretty much entirely on the frequency of use that a feature will get:
Keying up/down from title to body: Very common. Probably done on every single post written in the editor either via arrow keys or by enter key. == requirement
Copying and pasting all text+title in/out of the editor: Common-ish? Probably done by some people regularly and some people never at all. Having to copy title/body separately is a mild inconvenience for a non-primary usecase. == nice to have, not essential
Splitting title into body / joining body onto title: Rare. You really only do this if you've already made a mistake, or you end up in some weird situation where you somehow need to do this. This isn't a planned state which you will definitely end up in each time you use the editor. == edge case, not important
With everything like this, my question is: How often are people going to run into it? Is this something they need to inherently do with every post? or is it an edge case which is needed sometimes?

Cool, so the solution is the "nested mobiledocs" solution in the editor and then seperate the fields when they're saved in ghost-admin.
The problem is I have no idea how to do this as it was only ever mentioned casually in a conversation. I'll follow up with Cory and Mathew (who have both indicated that they're absolutely flat out for the next two weeks) to get pointed in the right direction.


[REL-382] Use Sentry's implementation of finishing with custom timestamp

Sentry added their own implementation that allowed setting a custom
timestamp for the end of a span in sentry-cocoa/pull/1993. However,
their implementation did not correctly support setting the timestamp
value on the top level SentryTracer span (aka: rootSpan) because
that passed through to the rootSpan and then the tracer only check if
the rootSpan had a timestamp set. If the rootSpan did have a
timestamp but the tracer itself had not been finished then the internal
finishing code would short circuit and never send to the server.
Note: I will submit this fix to SentryTracer to upstream separately.
Changes

Update SentryTracer finish logic to check the rootSpan for a
timestamp and that the context had a completion status set

Test Plan

Update Arc to use local check out of Sentry (or point the Package to
this commit)
Build Arc and enable Sentry instrumentation
Verify that results make it up to Sentry

Current dependencies on/for this PR:

development

PR #7   👈

This comment was auto-generated by Graphite.

Well, the change I made isn't likely to be correct outside of our use case (or at least there are more than a handful of tests that fail because of it).
I opened https://github.com/getsentry/sentry-cocoa/issues/2054 to track the issue upstream.


Catch some GCP auth errors

When we haven't auth'ed to GCP API, turbinia server and psqlworker will loop with:
Traceback (most recent call last):
  File "/home/romaing/venvs/turbinia-dev3/bin/turbiniactl", line 11, in <module>
    load_entry_point('turbinia==20181004', 'console_scripts', 'turbiniactl')()
  File "/home/romaing/venvs/turbinia-dev3/lib/python3.5/site-packages/turbinia-20181004-py3.5.egg/turbinia/turbiniactl.py", line 465, in main
    jobs_blacklist=args.jobs_blacklist, jobs_whitelist=args.jobs_whitelist)
  File "/home/romaing/venvs/turbinia-dev3/lib/python3.5/site-packages/turbinia-20181004-py3.5.egg/turbinia/client.py", line 761, in __init__
    self.task_manager.setup(jobs_blacklist, jobs_whitelist)
  File "/home/romaing/venvs/turbinia-dev3/lib/python3.5/site-packages/turbinia-20181004-py3.5.egg/turbinia/task_manager.py", line 116, in setup
    self._backend_setup(*args, **kwargs)
  File "/home/romaing/venvs/turbinia-dev3/lib/python3.5/site-packages/turbinia-20181004-py3.5.egg/turbinia/task_manager.py", line 397, in _backend_setup
    self.server_pubsub.setup_subscriber()
  File "/home/romaing/venvs/turbinia-dev3/lib/python3.5/site-packages/turbinia-20181004-py3.5.egg/turbinia/pubsub.py", line 86, in setup_subscriber
    self.subscriber.create_subscription(subscription_path, self.topic_path)
  File "/home/romaing/venvs/turbinia-dev3/lib/python3.5/site-packages/google/cloud/pubsub_v1/_gapic.py", line 40, in <lambda>
    fx = lambda self, *a, **kw: wrapped_fx(self.api, *a, **kw)  # noqa
  File "/home/romaing/venvs/turbinia-dev3/lib/python3.5/site-packages/google/cloud/pubsub_v1/gapic/subscriber_client.py", line 398, in create_subscription
    request, retry=retry, timeout=timeout, metadata=metadata
  File "/home/romaing/venvs/turbinia-dev3/lib/python3.5/site-packages/google/api_core/gapic_v1/method.py", line 143, in __call__
    return wrapped_func(*args, **kwargs)
  File "/home/romaing/venvs/turbinia-dev3/lib/python3.5/site-packages/google/api_core/retry.py", line 273, in retry_wrapped_func
    on_error=on_error,
  File "/home/romaing/venvs/turbinia-dev3/lib/python3.5/site-packages/google/api_core/retry.py", line 208, in retry_target
    time.sleep(sleep)
KeyboardInterrupt

it would be nice to have a check that we can auth to GCP, and get a specific error message

Yeah, this is related to this:
https://github.com/google/turbinia/issues/432
We trap these kinds of errors in some places but not others, so we need to make sure we do that everywhere.

I believe this is fixed now:
https://github.com/google/cloud-forensics-utils/blob/06d1367f0ac43d6af6248cbf56e11d1f70f4044d/libcloudforensics/gcp.py#L1100


Robert Longson: Watch allamericangrit.com

Robert Longson requests the watch of the watch_keyword allamericangrit\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

allamericangrit\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Cody Gray in Charcoal HQ


LAB 15 & 16 - Docker. Docker Image missed

Module: az-400
Lab/Demo: 15 & 16
Task: 00
Step: 00
Description of issue
In the Build task, from the CI pipeline, it looks for a microsft / aspnetcore: 1.0-2.0 image that no longer exists and prevents both labs from being able to start as soon as possible.
Repro steps:

My student, Carlos Lopez Perez, and I find a temporal solution:
In Repos-Files.
Go to docker-compose.ci.build.yml and change the image microsoft/aspnetcore by paultrampert/aspnetcore-build-with-yarn
The same in src-> MyHealth.Web-> Dockerfile
This image includes the .net core 3.0 sdk and allows the construction of the image necessary to perform both labs.
We hope it helps.

My student, Carlos Lopez Perez, and I, find a temporal solution.
Go to Repos -> Files And open docker-compose.ci.build.yml file.
Edit and change the image microsoft/aspnetcore:1.0-2.0 by paultrampert/aspnetcore-build-with-yarn
The go to Repos -> Files -> src folder -> MyHealth.Web Folder and open dockerfile file.
Edit and change FROM microsoft/aspnetcore:1.0-2.0 to FROM paultrampert/aspnetcore-build-with-yarn
This image includes the .net core 3.0 sdk and allows the construction of the image necessary to perform both labs.
We hope it helps.

Thanks Juan. It took a while to investigate where was the problem coming
from. I'm glad we worked it out and hopefully it will also help other
students.
Kind regards,
Carlos Pérez López
El vie., 16 jul. 2021 17:25, Juan Quijano @.***>
escribió:

My student, Carlos Lopez Perez, and I, find a temporal solution.
Go to Repos -> Files And open docker-compose.ci.build.yml file.
Edit and change the image microsoft/aspnetcore:1.0-2.0 by
paultrampert/aspnetcore-build-with-yarn
The go to Repos -> Files -> src folder -> MyHealth.Web Folder and open
dockerfile file.
Edit and change FROM microsoft/aspnetcore:1.0-2.0 to FROM
paultrampert/aspnetcore-build-with-yarn
This image includes the .net core 3.0 sdk and allows the construction of
the image necessary to perform both labs.
We hope it helps.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftLearning/AZ400-DesigningandImplementingMicrosoftDevOpsSolutions/issues/111#issuecomment-881530813,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIPSLOLDB5UPKTYLS7MCDD3TYBFPLANCNFSM5APAKXHA
.

All name changes are here 👉🏽 https://github.com/dotnet/dotnet-docker/issues/2375.
@JuanQuijano shouldn't we stick to official images instead of custom ones?

The issue was fixed with a static, outsourced image - not ideal, but at least the pipelines run again successfully. @krasmuss  Would you like to keep it that way or should we add steps to update to a newer, supported version?


Tower of Time: Very low performance compared to native version

Hi,
The game Tower of Time has a native Linux version (with an OpenGL renderer). However, since I experienced some crashes with the native version, I switched over to Proton/DXVK. Running it under DXVK produces rather awful performance with the average FPS being somewhere between 30-40 fps, and sometimes even struggling to meet a stable 30 fps.
The native Linux version with the OpenGL renderer hovers between 80-90 fps on the exact same settings. My hardware should obviously be able to run this game fairly well.
Software information
Tower of Time v1.4.5, stock "Very High" setting on 3440x1440 resolution.
The game runs on Unity 5.6.5 (exact version number unsure, this is according to PCGamingWiki).
System information

GPU: Radeon RX 6800 XT
Driver: Mesa 22.2.5 (RADV)
Wine version: Proton 7.0.6e
DXVK version: 1.10.3 (Bundled with Proton)

Apitrace file(s)
Not sure if apitrace files are useful for debugging performance issues, please let me know if this is helpful and I'll provide it. I technically have access to Windows (which I believe is preferred over Wine?) but it's a pain to get it up and running.
Log files
I've attached performance log files captured via mangohud. Both were taken at the exact same stock "very high" settings with the same scenes loaded. I can't upload these to flightlessmango.com because the login system doesn't work, sorry!
OpenGL:
TowerOfTime-OpenGL_2023-03-14_17-22-00.csv
TowerOfTime-OpenGL_2023-03-14_17-22-00_summary.csv
DXVK:
TowerOfTime-DXVK_2023-03-14_17-30-43.csv
TowerOfTime-DXVK_2023-03-14_17-30-43_summary.csv
Please let me know if there's anything else useful I can provide, I'll try to get the info ASAP.

Huh, that's very interesting! Indeed I'm running this on a 5900X, which has 24 threads. Oddly enough, I found that CPU usage was unexpectedly high during my playtime. Is this a common thing for Unity games?
I wasn't aware of the existence of WINE_CPU_TOPOLOGY (and I can't find any documentation for it either). Are there any downsides to using this as a workaround during my current playthrough?
Either way, I suppose this would all make this more of a Wine issue than a DXVK issue, correct?

I wasn't aware of the existence of WINE_CPU_TOPOLOGY (and I can't find any documentation for it either). Are there any downsides to using this as a workaround during my current playthrough? I'll try it out on my system as soon as I can.

There isn't really any downside if it's faster.

Either way, I suppose this would all make this more of a Wine issue than a DXVK issue, correct?

No, this is usually a problem with the game itself.

No, this is usually a problem with the game itself.

So is the expectation then that this game would have similar performance results on Windows? I can make some time and check it out on Windows before I close the issue if that's helpful.

I've observed a similar issue in a few games with 1700X and 5800X cpu's. For some years I run wine games with "taskset -c 0-7" for better performance.
WINE_CPU_TOPOLOGY is new to me as well.

WINE_CPU_TOPOLOGY is new to me as well.

It's a Proton Wine thing https://github.com/ValveSoftware/wine/blob/492470267af486cb24ad0deaa028168eb94618a4/dlls/ntdll/unix/system.c#L555

taskset will not help if the game does stupid stuff depending on how many cpu core it thinks there are, because taskset doesn't change the number of cores reported to the game
WINE_CPU_TOPOLOGY is only in proton
some games have problems with high reported core counts only on wine, can't remember which, but I do remember that there are some that need it only on linux. But I would assume that's more of a wine than dxvk problem.

taskset will not help if the game does stupid stuff depending on how many cpu cores it thinks there are, because taskset doesn't change the number of cores reported to the game

WINE_CPU_TOPOLOGY is only in proton

some games have problems with high reported core counts only on wine, can't remember which, but I do remember that there are some that need it only on linux. But I would assume that's more of a wine than dxvk problem.

Unity games tend more to run into this issue afaik

So is the expectation then that this game would have similar performance results on Windows? I can make some time and check it out on Windows before I close the issue if that's helpful.

@romatthe Windows applies a similar hack for some games.


feat: output can be stdout so you can pipe it to other commands

output now accepts '-' and sends the output to stdout so you can pipe it to other commands.

@rap2hpoutre what do you think of this PR?

Thank you for your contribution!
Why did you change all console.log to console.error? It could seem weird to console.error a "process started" info.

I had to switch the logging statements to STDERR because the new feature is output to STDOUT, and otherwise the log statements would end up in STDOUT and potentially mess up the resulting pg_dump output.

@rap2hpoutre If you prefer me to do it differently, let me know! It's easy enough to change.

@rap2hpoutre does it look good to you?

I had to switch the logging statements to STDERR because the new feature is output to STDOUT, and otherwise the log statements would end up in STDOUT and potentially mess up the resulting pg_dump output.

That's OK then!

@rap2hpoutre does it look good to you?

Sorry for late answer! Yes we can merge it.
Thank you for your work!!!


Ingress rules for httpbin service not working

After publishing Ingress rule for httpbin service is accessible at "/" instead of "/httpbin"
Ingress pod shows following warning in pod log
[2017-11-14 12:15:36.874][128][warning][http] src/envoy/mixer/mixer_control.cc:419] Could not convert to ip: destination.ip:
[2017-11-14T12:15:36.874Z] "GET /headers HTTP/1.1" 200 - 0 822 10 7 "<IP_ADDRESS>" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.89 Safari/537.36" "9e215ec9-7331-9b92-8b8d-e26e660b0bb3" "<IP_ADDRESS>" "<IP_ADDRESS>:8000"

:~$ istioctl version
Version: 0.2.12
GitRevision: 998e0e00d375688bcb2af042fc81a60ce5264009
GitBranch: release-0.2
User: releng@0d29a2c0d15f
GolangVersion: go1.8.3

:~$ kubectl version
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.0", GitCommit:"6e937839ac04a38cac63e6a7a306c5d035fe7b0a", GitTreeState:"clean", BuildDate:"2017-09-28T22:46:41Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.0", GitCommit:"6e937839ac04a38cac63e6a7a306c5d035fe7b0a", GitTreeState:"clean", BuildDate:"2017-09-28T22:46:41Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}

$ kubectl describe svc httpbin
Name:              httpbin
Namespace:         default
Labels:            app=httpbin
Annotations:       kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"httpbin"},"name":"httpbin","namespace":"default"},"spec":{"ports":[{"...
Selector:          app=httpbin
Type:              ClusterIP
IP:                <IP_ADDRESS>
Port:              http  8000/TCP
TargetPort:        8000/TCP
Endpoints:         <IP_ADDRESS>:8000
Session Affinity:  None
Events:            <none>

$ kubectl get  pods httpbin-6cf5fb67d-sjkzg -o wide
NAME                      READY     STATUS    RESTARTS   AGE       IP            NODE
httpbin-6cf5fb67d-sjkzg   2/2       Running   0          3h        <IP_ADDRESS>

$ kubectl describe ingress simple-ingress
Name:             simple-ingress
Namespace:        default
Address:          <IP_ADDRESS>
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host  Path  Backends
  ----  ----  --------
  *
        /.*   httpbin:8000 (<none>)
Annotations:
Events:  <none>

I can confirm I encountered the same:
[2017-11-22 04:14:22.729][259][warning][http] src/envoy/mixer/mixer_control.cc:419] Could not convert to ip: destination.ip:
[2017-11-22T04:14:22.729Z] "GET / HTTP/1.1" 200 - 0 11602 13 9 "<IP_ADDRESS>, <IP_ADDRESS>" "curl/7.29.0" "5551708a-a381-9630-8537-fbba21aac605" "acc4a702ccda411e7bd6a0ab14db3372-2086655442.ap-southeast-2.elb.amazonaws.com" "<IP_ADDRESS>:8000"

This is pretty stale. Can you retest with 0.8