opennebula deletion of logdb table is not chunked and delays startup

Bug Report
Version of OpenNebula

[ ] 5.2.2
[ ] 5.4.0
[ ] 5.4.1
[ ] 5.4.2
[ ] 5.4.3
[ ] 5.4.4
[ ] 5.4.5
[x] 5.4.6
[ ] 5.4.7
[ ] Development build

Component

[ ] Authorization (LDAP, x509 certs...)
[ ] Command Line Interface (CLI)
[ ] Contextualization
[ ] Documentation
[ ] Federation and HA
[ ] Host, Clusters and Monitorization
[ ] KVM
[ ] Networking
[ ] Orchestration (OpenNebula Flow)
[ ] Packages
[ ] Scheduler
[ ] Storage & Images
[ ] Sunstone
[ ] Upgrades
[ ] User, Groups, VDCs and ACL
[ ] vCenter

Not sure the exact component related but this is related to LogDB::purge_log() which I think is called on startup.
Description
On startup colleagues have seen that opennebula seems to hang waiting on the database.
Further investigation showed that it was trying to delete 23,000,000 rows from the logdb table
in a single transaction. Supposedly auto-purging is enabled (I'm told). I'm the DBA that manages
this database.
This took the MySQL server about 15 minutes to execute and resulted in 150MB of binlogs being generated (using minimal RBR) on mysql-community-server-5.7.18. The full binlogs would have been much larger.
Expected Behavior
It seems to me that a few things are being done incorrectly here:

do not do large transactions but delete rows in smaller chunks (even if you have to delete this number of rows)
delete in small chunks giving a small delay between each one, thus not consuming all I/O on the server
background this process as I guess it shouldn't affect other tasks that are ongoing and would allows the processes to start up with an interruption.

Actual Behavior

one potentially huge DELETE statement.

How to reproduce
Startup opennebula and see that it seems to hang for a long time.

requires having a large unpurged logdb table.

For many large busy MySQL systems where replication may be involved it's convenient always to break large transactions into smaller chunks.
My suggestion would be to:

run the DELETE statement in a loop with a LIMIT X (where X is configurable but something like 10000 by default)
record the number of rows deleted. Most DB drivers provide this information. I'm not sure if the interface you are using provides it.
add a small configurable delay (2ms?) if the number of rows deleted > 0
exit the loop when number of deleted rows  = 0

Related to: https://github.com/OpenNebula/one/issues/1365

Thank you very much for the detailed information. We are going to follow the proposed approach to purge log entries in small chunks.
We'll address this issue by:

Adding a configuration parameter with the max number of log rows to delete every purge timeout
Limiting the delete query to this max number of log rows.

Merged upstream.


zip.vim can not extract file - shell not defined

Steps to reproduce

run vim some.zip
select a file in the zip
hit x to extract the file
Error occurs:
Error detected while processing function zip#Extract:
line   24:
E121: Undefined variable: shell
E116: Invalid arguments for function shellescape(shell))
E116: Invalid arguments for function system
error attempted to extract  but it doesn't appear to be present!

Expected behaviour
the selected file gets extracted from the zip
Version of Vim
VIM - Vi IMproved 9.0 (2022 Jun 28, compiled Jun 29 2024 05:34:37) macOS version - arm64 Included patches: 1-2136, 2142
Environment
OS: macOS 14.6.1
$TERM: xterm-256color
Shell: both fish and bash
Logs and stack traces
No response

that is an old version. Can you please try with latest version?

it works with
VIM - Vi IMproved 9.1 (2024 Jan 02 kompiliert am Aug 01 2024 20:48:53)
MacOS Version - arm64
Inklusive der Patches: 1-650

but it fails the same way with
NVIM v0.10.1
Build type: Release
LuaJIT 2.1.1724512491

which is the latest one. I know that here is not the nvim repo, but they seem to use the same zip.vim as vim. What have been the changes to fix the behavior in vim?

Well, you can compare the runtime/plugin/zipPlugin.vim and runtime/autoload/zip.vim for both repos. Or run
git log -- runtime/plugin/zipPlugin.vim runtime/autoload/zip.vim

in the top level of this repository.


Enhance Trailer Download

I used a youtube link extractor to extract a downloadable youtube video link then used the Android Download Manager to download the file to a folder named "BigUp".
I hope it meets your expectations.

Ok then, Review it anytime


(ISSUE CLOSED)

Closed. Please delete lmao

there are many reasons to get this error.

if you are testing with simulator, generally there are very limited ads to display, so we will get this error. but this will work in production.
If you have created your ad units newly, so that will take some time to become active for serving ads, generally it will take 3 - 5 days to get active, so please wait and have fun.
there will be some cases that you will keep some restrictions on serving ads like only display amount more than certain amount, that time ads may not be available for serving the same. so keep your ad settings always in an optimised manner.

i think these are the possible causes. let me know if you need any other info

Sorry for the informal close of the issue, the owners of the repo can probably delete this.
It has been a while since the issue has been opened, and I have changed to a different framework. Thanks for the help anyway!


Escaping a ] inside a posix class [[:\]:] does funky stuff

The regex [[:upper:]] works as expected at matching only upper case letters.
Normally if you missspell anything inside the posix class, the textmate engine will fail. Except for a few noteble expcetions; ], : and [: (only gonna focus on ] for now)
[[:upp]er:]] will work with matching :], u], p], e] and r]. (regex: [:uper]\]).
But if you escape the ] inside the posix class, then both square brackets acts as both the closing and opening brackets for the 2nd character class and as literary characters for the 1st class. Leaving the last ] competely out of the classes (and can be removed without error).
[[:upp\]er:]] will match the same as above but also with the added [ and ].
:], u], p], e] r] [] and ]]. (regex:[\[:uper\]]\]).
[[:\]:] will match [, : and ].
~Placing a - before the last : causes textmate engine to fail [[:\]-:].~ (EDIT: \]-: is an illegal character range)
Regex works as expected if either : is removed or moved one space away from their respected square bracket.
(I have used a single \ instead of the required double \\ from json embeding)

Posix classes are some of the funkiest things in textmate.
The below regex is allowed and will match all upper case letters aswell as a, b and c.

But if posix a value is inputted, the engine breaks:

kkos has fixed this issue
https://github.com/kkos/oniguruma/commit/d1cf59269006481294896fb855153857d8bec928
https://github.com/kkos/oniguruma/commit/9b9365d0524d1bf60c1fc963be13260ff6b798aa
update version of oniguruma?
1.5.1 => 6.9.6

VSCode is still using an old 2020 version
https://github.com/kkos/oniguruma/releases/tag/v6.9.5_rev1
https://github.com/microsoft/vscode/blob/7cbff1919e05c97cc95e06a32abbe7ab8d3f0fc3/src/vs/workbench/services/textMate/common/cgmanifest.json#L9
@hediet


redirect url issue

Getting this issue everytime
Authorization failed Error Domain=LIALinkedInERROR Code=1 "The operation couldn’t be completed. (LIALinkedInERROR error 1.)

LinkedIn changed the conditions in the usage of their API´s recently (see the top of the readme.md).
You could also see this issue for a suggested solution:
https://github.com/jeyben/IOSLinkedInAPI/issues/59


Can't configure additional dictionaries

Adding additional license dictionaries doesn't work as intended (at least not for me). The docs say that you can provide a list of dictionaries but what you actually expect is a list of files, where each file has a single line with the name of a file that holds a dictionary. See https://github.com/merkle-open/gradle-license-enforce-plugin/blob/bc2326d1bae1a2e583d3336f869a11665a3f708e/src/main/kotlin/com/namics/oss/gradle/license/LicenseEnforceTask.kt#L167
IMO, line 167 should be removed.
I tried to work around the issue by using an extra file in between but I couldn't get it to work; I tried using absolute paths, both windows and unix paths, quotes.... It always fails with :
Caused by: org.gradle.api.UncheckedIOException: java.io.FileNotFoundException: additional-license-definitions.yaml
 (The filename, directory name, or volume label syntax is incorrect)
...
Caused by: java.io.FileNotFoundException: additional-license-definitions.yaml
 (The filename, directory name, or volume label syntax is incorrect)
	at kotlin.io.FilesKt__FileReadWriteKt.readText(FileReadWrite.kt:125)
	at kotlin.io.FilesKt__FileReadWriteKt.readText$default(FileReadWrite.kt:125)
	at com.namics.oss.gradle.license.LicenseEnforceTask.addConfiguredDictionaries(LicenseEnforceTask.kt:167)
	at com.namics.oss.gradle.license.LicenseEnforceTask.initializeDictionary(LicenseEnforceTask.kt:148)
	at com.namics.oss.gradle.license.LicenseEnforceTask.enforce(LicenseEnforceTask.kt:59)

I think the issue might be that you don't trim the line, but that's irrelevant because, as I wrote above, I think you need to remove line 167.

Hi @theseion I applied your suggestion next to some other changes. Does this fix work for you?

Thanks. I will have to check. I'll probably get around to it next week.


Receiving a push notification raises a Java exception on my Samsung phone

In DW.FCMManager.Android.pas, the call to presentNotification() is raising a Java exception which is crashing my app.
Delphi 12.1, running the latest Kastri sources on a Samsung Note 20, Android 13.
procedure TPlatformFCMManager.MessageReceivedNotificationHandler(const Sender: TObject; const AMsg: TMessage);
var
  LIcon: string;
  LIntent: JIntent;
begin
  if ShowBannerIfForeground and IsForeground then
  begin
    LIntent := TMessageReceivedNotification(AMsg).Value;
    TOSMetadata.GetValue(cMetadataFCMDefaultNotificationIcon, LIcon);
    TJDWNotificationPresenter.JavaClass.presentNotification(TAndroidHelper.Context, LIntent, StringToJString(FChannelId), StrToIntDef(LIcon, 0));
  end;
end;

The error is:
java.lang.IllegalArgumentException: Invalid notification (no valid small icon): Notification(channel=MYSHIFTPLANNERv1 shortcut=null contentView=null vibrate=null sound=null defaults=0x0 flags=0x10 color=0x00000000 vis=PRIVATE semFlags=0x0 semPriority=0 semMissedCount=0)

Just before it, in the logcat output are these, which may be relevant:
Key com.google.firebase.messaging.default_notification_icon expected String but value was a java.lang.Integer. The default value <null> was returned.

and
DWNotificationPresenter	Unable to locate resource notification_custom

DWNotificationPresenter	Unable to locate resource notification_custom

You can ignore this message - it just indicates that you're not using custom notifications
Key com.google.firebase.messaging.default_notification_icon expected String but value was a java.lang.Integer. The default value <null> was returned.

This one is a bit odd - can you show what you have for it in AndroidManifest.xml?

Yes, this one in particular:
<meta-data android:name="com.google.firebase.messaging.default_notification_icon" android:value="" android:resource="@drawable/ic_notification"/>

I've now realized that this line of code might be incorrect:
  TOSMetadata.GetValue(cMetadataFCMDefaultNotificationIcon, LIcon);

Since LIcon is a string, and I expect if the resource is actually there, it's really an integer. Not sure why this hasn't been a problem before. Can you try changing the code so that LIcon is an Int64, thus meaning that it can be passed direct to presentNotification instead of using StrToIntDef.
Also, when including multiple lines of code on comment, you can use 3 backticks () at the beginning and end, and it also accepts a format specification for supported types at the beginning marker, like: xml and ```delphi etc.

Thanks Dave, I'll try that and report back.
I can never remember the correct markdown syntax for code so thanks for the reminder. I used 2 backticks so almost there!

Same error I'm afraid.
First chance exception at $0000006D7E6F474C. Exception class EJNIException with message 'java.lang.IllegalArgumentException: Invalid notification (no valid small icon): Notification(channel=MYSHIFTPLANNERv1 shortcut=null contentView=null vibrate=null sound=null defaults=0x0 flags=0x10 color=0x00000000 vis=PRIVATE semFlags=0x0 semPriority=0 semMissedCount=0)'. Process myShiftPlanner.apk (23820)

to confirm, the changed method is
procedure TPlatformFCMManager.MessageReceivedNotificationHandler(const Sender: TObject; const AMsg: TMessage);
var
  LIcon: Int64;
  LIntent: JIntent;
begin
  if ShowBannerIfForeground and IsForeground then
  begin
    LIntent := TMessageReceivedNotification(AMsg).Value;
    TOSMetadata.GetValue(cMetadataFCMDefaultNotificationIcon, LIcon);
    TJDWNotificationPresenter.JavaClass.presentNotification(TAndroidHelper.Context, LIntent, StringToJString(FChannelId), LIcon);
  end;
end;

Perhaps you have no valid small icon? ;-) I'll see if I can find out what the issue is...

I've got them all listed in the project, so unless it's getting it from somewhere else then not sure what I'm looking for!

It may be that the actual icons are not being deployed? One sure way to check is to look at the project output folder, e.g.:

This is for a project that was probably created with Delphi 11.x and later "upgraded" to 12.x, so I may be missing files, however those that Delphi is deploying work on my Pixel 6 Pro - I suspect you have either no ic_notification.png files being deployed, or missing the one your device needs.

I checked my deployment settings, output folder etc and it all seems correct. ic_notification.png is in the correct res folders, correct size and none are corrupt. Everything it needs seems to be there, so must be something else causing it not to be able to find the icons at runtime.
Your demo project seems to work fine on my phone too, so must be something in my app setup but no idea what it could be.
I'll keep digging. 👍
Thanks

Sorry, I meant to comment here a few days ago when I pushed a fix. Can you check whether it works for you now?


fix: Fix doc syntax errors

Fix doc syntax errors

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Incognito Mode

Is there any way I can open the browsers in incognito mode? I don't want to save cache/cookies, preferably on iOS

Hi @zanelladev,
Unfortunately, there is no way to open a browser in incognito mode using this plugin.
The platform implementation does not provide an option for this.


How to use this project in Vue ?

Hi author, this project is great, but I encountered a problem which is this project can't work in Vue. Can this project use in Vue？

Hmm, I have not tried it, but I do not see why not.
You will want to add Openseadragon src before your main.js for vue, then call Opeanseadragon in the mounted() lifecycle

It works. Thank you

I personally created a component for this exact purpose:
Vue.component('panorama', {
    props: ['dzi'],
    data: function() {
        return {
            viewer: null,
        }
    },
    template: `<div class="panorama" ref="container"></div>`,
    mounted() {
        this.viewer = OpenSeadragon({
            element: this.$refs.container,
            prefixUrl: '/static/openseadragon/images/',
            tileSources: this.dzi
        })
    },
})

The usage is that simple:
<panorama dzi="/path/to/dzi"></panorama>

Anyone have this working with Vue 3?


error: load xxx.lua

(emqttd@<IP_ADDRESS>)1> 13:05:33.233 [error] Lua Hook: Failed to load lua script "hook_lua/test.lua" due to error {undef,[{luerl_scan,string,["\nfunction on_message_publish(clientid, username, topic, payload, qos, retain)\n    return topic, "hello", qos, retain\nend\n\nfunction register_hook()\n    return "on_message_publish"\nend\n\n\n"],[]},{luerl_comp,do_read_file,1,[{file,"src/luerl_comp.erl"},{line,176}]},{luerl_comp,do_passes,2,[{file,"src/luerl_comp.erl"},{line,133}]},{luerl_comp,compile,2,[{file,"src/luerl_comp.erl"},{line,98}]},{luerl,loadfile,2,[{file,"src/luerl.erl"},{line,95}]},{luerl,dofile,2,[{file,"src/luerl.erl"},{line,72}]},{emq_lua_hook_cli,do_load,1,[{file,"src/emq_lua_hook_cli.erl"},{line,177}]},{emq_lua_hook_cli,'-do_loadall/0-lc$^0/1-0-',1,[{file,"src/emq_lua_hook_cli.erl"},{line,173}]}]}

What's your emqttd version? A recent bug fix has solved similar problem. Please refer to this week's official release.

hi,
wget http://emqtt.com/downloads/latest/ubuntu14_04
OS: Ubuntu 14.04
Erlang/OTP: 19
EMQ: Latest (2.2 beta)
Nothing has changed,  same before.

@chenqy2018  Sorry for the confusion. The version that fixes this problem should be 2.2beta.2 on this friday. And the version you test is 2.2beta.1.

It's work,  thank you.


(Bug?): getCellNode Not Returning Correct Rows in Latest Version (Regression from 8.1.5)

In the latest version of Serenity.Corelib (version 8.8.1), the method in grid.ts, of sleekgrid, getCellNode(row, cell) https://github.com/serenity-is/sleekgrid/blob/main/src/grid/grid.ts#L3827 does not return the expected DOM node for rows that are outside the currently rendered range. This issue was not present in version 8.1.5, where getCellNode reliably returned the requested rows regardless of their position in the grid.
Steps to Reproduce

Initialize a grid with a large dataset (enough to enable virtual scrolling).
Call getCellNode(row, cell) for a row that is currently not visible (e.g., scrolled out of view).
Observe that getCellNode returns null or an unexpected value.
Expected behavior in version 8.1.5:

getCellNode(row, cell) returns the DOM node even if the row is not currently rendered.

Behavior in the latest version:

getCellNode(row, cell) returns null for rows that are not rendered.

Impact
This issue impacts functionalities relying on dynamically interacting with rows that are not currently rendered (e.g., background updates, programmatically selecting rows, etc.). I use it to render printing for all data for reports (we cannot use it with pagination due to grouping summaries)
Screenshot from my local:

Workaround Attempted
I've tried forcing the grid to render missing rows by manually entering renderAllCells and renderAllRows to true, but this is inefficient and causes performance issues, especially for large datasets.
Additional Information
This issue seems to be related to how the cache (_rowsCache) is maintained in the latest version. It appears that rows outside the visible range are no longer being cached, unlike in 8.1.5. However, I am not sure if that was the case.
Request for Help
Could you provide guidance on how to ensure that getCellNode(row, cell) works as expected for all rows, not just the ones currently rendered and having virtualization enabled at the same time?
Thank you for your time and support!

getCellNode never returned cells outside view, maybe some buffer before after view. It is how virtualization works. Use renderAllRows and auto height for reports.

My apologies, it appeared that I missed scrolling _grid.scrollRowToTop(r); while getting _grid.getCellNode(r, c) in slick.pring.js plugin. Anyway, Thank you! :)

Ehh, I'm so sorry to bug you again, @volkanceylan. However, I just experienced this issue when I enabled virtualization.

I used this code plugin, line 35 is bugging sleekGrid.
https://github.com/wmluke/slickgrid-print-plugin/blob/master/src/scripts/slickgrid-print-plugin.js
It used to work with 8.1.5, is there anything that I can replace to make it work?

I found out that I can replace sanitizer option which fixed the issue.
grid.setOptions({ sanitizer: (dirtyHtml) => dirtyHtml });
Thank you!


Add policy note about re-opening old bikesheds.

cc @rust-lang-nursery/style
We decided on adding this note in the first meeting. Let me know if you have a better wording or think I should place the note elsewhere.

Looks good to me.

Thanks!


Add new projects in the EXTRA PROJECTS folder

Add a complete description of the project.
And add the code for the project.
It would be great to have a live link for the project.
Before doing anything checkout the contributing.md file.
Link the issue in the PR.

Hi @MohammedHamzaMalik, I want to contribute.
I have this project https://music-play-peach.vercel.app/
It is a music player

Yeah sure @UditNS

wow i love this project well done you have done such an amazing job ✌
all my favourite songs

hello guys i just wanted to join this challenge of 100 days javascript .
can you please give me some information about the challenge and how can i share my projects here 😇

Your Project is so great


Rainloop doesn't work with virtualhost

I am trying to call my rainloop instance from a virtualhost but using subdirectories.
rsanchez@babylon:/etc/nginx/vhosts$ more rainloop.conf
location ^~ /webmail {
    root /srv/rainloop/public_html;
    try_files $uri $uri/ /webmail/index.php?$query_string;
    #if (!-e $request_filename) { rewrite ^ /webmail/index.php last; }
    access_log /srv/rainloop/logs/access.log;
    error_log /srv/rainloop/logs/error.log;
    index index.php;
    access_log /var/log/nginx/scripts.log scripts;

    location ~ \.php$ {
        #if (!-f $request_filename) { return 404; }
        include fastcgi_params;
        #fastcgi_split_path_info ^(.+\.php)(/.+)$; #this line
        fastcgi_pass unix:/var/run/php5-fpm.sock;
        fastcgi_index index.php;
        #fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param SCRIPT_FILENAME /srv/rainloop/public_html/index.php;
    }

    location ~ /\.ht {
        deny all;
    }

    location ^~ /webmail/data {
        deny all;
    }

    #location ~ /webmail/\.css {
    #    root /srv/rainloop/public_html;
    #    add_header  Content-Type    text/css;
    #}

    #location ~ /webmail/\.js {
    #    root /srv/rainloop/public_html;
    #    add_header  Content-Type    application/x-javascript;
    #}
}

The main file is /etc/nginx/sites-available/default. From here I call the code from above.
upstream zboss {
        server unix:/srv/zboss/run/zboss.sock fail_timeout=0;
}

server {
        listen 80 default_server;
        server_name babylon;
        root /var/www/html;

        location / {
                root /var/www/html;
                index index.html;
        }

        location /static {
                alias /srv/zboss/static;
        }

        location /prot/media {
                alias /var/www/uploads/zboss/;
        }

        location = /favicon.ico { access_log off; log_not_found off; }
        location /prot {
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
                proxy_set_header SCRIPT_NAME /prot;
                proxy_redirect off;
                if (!-f $request_filename) {
                        proxy_pass http://zboss;
                        break;
                }
        }

        location /wiki {
                include uwsgi_params;
                uwsgi_param SCRIPT_NAME /wiki;
                #uwsgi_pass unix:///usr/local/share/moin/moin.sock;
                uwsgi_pass <IP_ADDRESS>:8080;
                uwsgi_modifier1 30;
        }

        location /cgit {
                proxy_set_header X-Real-IP  $remote_addr;
                proxy_set_header X-Forwarded-For $remote_addr;
                proxy_set_header Host $host;
                proxy_pass http://<IP_ADDRESS>:85;
        }

        include vhosts/*.conf;
}

When trying http://babylon/webmail, I get a blank page. Checking the code I see that my code cannot load http://babylon/rainloop/v/<IP_ADDRESS>/static/js/min/boot.min.js?standard
When I see the nginx logs, I found:
2017/09/21 15:32:08 [error] 1737#0: *31774 open() "/var/www/html/rainloop/v/<IP_ADDRESS>/static/js/min/boot.min.js" failed (2: No such file or directory), client: <IP_ADDRESS>, server: babylon, request: "GET /rainloop/v/<IP_ADDRESS>/static/js/min/boot.min.js?standard HTTP/1.1", host: "babylon", referrer: "http://babylon/webmail"
But, "/var/www/html/rainloop/v/<IP_ADDRESS>/static/js/min/boot.min.js" is not the real location. The real location is "/srv/rainloop/public_html/rainloop/v/<IP_ADDRESS>/static/js/min/boot.min.js"
My real problem is then to change: /var/www/html for /srv/rainloop/public_html/
How to solve this?

Hi, I had a similar issue with ownCloud not physically in the web root path. I did not get that to work like this:
server {
  root /srv/www/vhosts/main_site/htdocs/;
[...snip...]
location ^~ /owncloud {
  root /var/www/owncloud/;
[...snip...]

So I commented that root line for the owncloud location and created a symlink for owncloud in /srv/www/vhosts/main_site/htdocs/ to /var/www/owncloud and that worked. Perhaps this will work for you. If you do find out how to work with a different root for specific locations let me know :)

I fixed it by adding...
location /rainloop/ {
   alias /srv/rainloop/public_html/rainloop/;
}

However I have 2 files that are linked incorrectly:
http://babylon/?/Css/0/User/-/Default/-/78eff971280337d6870e55d2c248037e/Hash/-/
<link type="text/css" rel="stylesheet" href="./?/Css/0/User/-/Default/-/78eff971280337d6870e55d2c248037e/Hash/-/" id="app-theme-link">
http://babylon/?/AppData@no-mobile-0/0/2853356928851618/
<script type="text/javascript" data-cfasync="false" src="./?/AppData@no-mobile-0/0/20454895033656695/"></script>
They should be:
http://babylon/webmail/?/Css/0/User/-/Default/-/78eff971280337d6870e55d2c248037e/Hash/-/
http://babylon/webmail/?/AppData@no-mobile-0/0/2853356928851618/
How can I fix them?


doesn't automaticly install required packages in the venv

ive tried running build_app.bat,
manually installing setuputils,
but then i get the error:
Collecting colorharmonies==1.0.5 (from -r requirements.txt (line 1))
  Using cached colorharmonies-1.0.5-py3-none-any.whl (3.6 kB)
Collecting colour==0.1.5 (from -r requirements.txt (line 2))
  Using cached colour-0.1.5-py2.py3-none-any.whl (23 kB)
Collecting contourpy==1.0.5 (from -r requirements.txt (line 3))
  Using cached contourpy-1.0.5.tar.gz (12.2 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting CTkMessagebox==2.1 (from -r requirements.txt (line 4))
  Obtaining dependency information for CTkMessagebox==2.1 from https://files.pythonhosted.org/packages/3b/10/16187d90643dfd5a5f84dfc141c8627da3ec2a372ece5a33823c93074f72/CTkMessagebox-2.1-py3-none-any.whl.metadata
  Using cached CTkMessagebox-2.1-py3-none-any.whl.metadata (7.2 kB)
Collecting customtkinter==5.2.1 (from -r requirements.txt (line 5))
  Obtaining dependency information for customtkinter==5.2.1 from https://files.pythonhosted.org/packages/d9/22/9015be3112284c6bc5ed2ad2122d273a3c2242d5e6cbb121e027d1c77004/customtkinter-5.2.1-py3-none-any.whl.metadata
  Using cached customtkinter-5.2.1-py3-none-any.whl.metadata (652 bytes)
Collecting cycler==0.11.0 (from -r requirements.txt (line 6))
  Using cached cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting darkdetect==0.8.0 (from -r requirements.txt (line 7))
  Using cached darkdetect-0.8.0-py3-none-any.whl (9.0 kB)
Collecting fonttools==4.38.0 (from -r requirements.txt (line 8))
  Using cached fonttools-4.38.0-py3-none-any.whl (965 kB)
Collecting importlib-resources==5.12.0 (from -r requirements.txt (line 9))
  Using cached importlib_resources-5.12.0-py3-none-any.whl (36 kB)
Collecting kiwisolver==1.4.4 (from -r requirements.txt (line 10))
  Using cached kiwisolver-1.4.4.tar.gz (97 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting loguru==0.7.2 (from -r requirements.txt (line 11))
  Obtaining dependency information for loguru==0.7.2 from https://files.pythonhosted.org/packages/03/0a/4f6fed21aa246c6b49b561ca55facacc2a44b87d65b8b92362a8e99ba202/loguru-0.7.2-py3-none-any.whl.metadata
  Using cached loguru-0.7.2-py3-none-any.whl.metadata (23 kB)
Collecting matplotlib==3.7.1 (from -r requirements.txt (line 12))
  Using cached matplotlib-3.7.1.tar.gz (38.0 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting numpy==1.23.4 (from -r requirements.txt (line 13))
  Using cached numpy-1.23.4.tar.gz (10.7 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [33 lines of output]
      Traceback (most recent call last):
        File "C:\Users\58147793\ctk_theme_builder\venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\58147793\ctk_theme_builder\venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\58147793\ctk_theme_builder\venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 112, in get_requires_for_build_wheel
          backend = _build_backend()
                    ^^^^^^^^^^^^^^^^
        File "C:\Users\58147793\ctk_theme_builder\venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 77, in _build_backend
          obj = import_module(mod_path)
                ^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\58147793\AppData\Local\Programs\Python\Python312\Lib\importlib\__init__.py", line 90, in import_module
          return _bootstrap._gcd_import(name[level:], package, level)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
        File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
        File "<frozen importlib._bootstrap_external>", line 994, in exec_module
        File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
        File "C:\Users\58147793\AppData\Local\Temp\pip-build-env-no3t5hdl\overlay\Lib\site-packages\setuptools\__init__.py", line 16, in <module>
          import setuptools.version
        File "C:\Users\58147793\AppData\Local\Temp\pip-build-env-no3t5hdl\overlay\Lib\site-packages\setuptools\version.py", line 1, in <module>
          import pkg_resources
        File "C:\Users\58147793\AppData\Local\Temp\pip-build-env-no3t5hdl\overlay\Lib\site-packages\pkg_resources\__init__.py", line 2172, in <module>
          register_finder(pkgutil.ImpImporter, find_on_path)
                          ^^^^^^^^^^^^^^^^^^^
      AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Which version of Python do you have installed?
Which version of Windows?

windows 10, version 22H2, os build 19045.3930
python 3.11.7 but i also have 3.12.1 installed the "py" and "python" commands automaticly use 3.12.1 but i have it set up that "python3117" wil use 3.11.7

also i just noticed when i try to change the widget geometry an empty toplevel pops up and i get the error
Traceback (most recent call last):
  File "C:\Users\58147793\AppData\Local\Programs\Python\Python312\Lib\tkinter\__init__.py", line 1962, in __call__
    return self.func(*args)
           ^^^^^^^^^^^^^^^^
  File "C:\Users\58147793\AppData\Local\Programs\Python\Python312\Lib\site-packages\customtkinter\windows\widgets\ctk_button.py", line 554, in _clicked
    self._command()
  File "C:\Users\58147793\ctk_theme_builder\lib\control_panel_v.py", line 948, in <lambda>
    command=lambda: self.launch_widget_geometry('CTkOptionMenu'))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\58147793\ctk_theme_builder\lib\control_panel_v.py", line 1054, in launch_widget_geometry
    geometry_dialog = GeometryDialog(master=self, widget_type=widget_type,
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\58147793\ctk_theme_builder\lib\geometry_dialog_v.py", line 25, in __init__
    self.enable_tooltips = mod.preference_setting(db_file_path=DB_FILE_PATH, scope='user_preference',
                           ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'lib.ctk_theme_builder_m' has no attribute 'preference_setting'```

i start by unzipping theme_builder_setup.py from ctk_theme_builder-2.5.1.zip
i run python3117 theme_builder_setup.py -p "C:\Users\58147793\Downloads\ctk_theme_builder-2.5.1.zip"
usually the program would freeze here for more than an hour (thats when i decided to terminate it and run build_app.bat manually to see the output)
but today it just worked and installed the packages in the venv

I managed to get Python 3.11 installed on my development  box yesterday, and I noticed that error appear. I am not sure why it didn't appear before, but I have fixed it.  I intend to continue working this through, but I may not get any updates released until the weekend.
In the meantime, please supply the logs, there should be at least 2 in the log directory, as described in my previous post.

i think you are right
PS C:\Users\58147793\ctk_theme_builder> .\venv\scripts\activate
(venv) PS C:\Users\58147793\ctk_theme_builder> python --version
Python 3.12.1```

So, it looks like the path to your Python 3.12 installation, appears on your Windows %PATH% variable before your Python 3.11 install. You could confirm this by typying echo %PATH%, and pressing enter. This means that Python 3.12 is getting seeded to the virtual environment folder (venv). You could uninstall Python 3.12, and remove the ctk_theme_builder directory where you installed to and then retry the install process. The other option, is to try this:
ctk_theme_builder-3.0.0.zip

Remove the old ctk_theme_builder directory;
Extract the theme_builder_setup.py file;
Run the install as described previously.

The 3.0.0 file, includes an update of all the libraries. It installs aginst Python 3.12 on Linux Mint, just fine. I haven't tried it on Windows, and so that's where you come in. Please let me know how you get on.

I will try that  but i just noticed that you asked for the logs. Do you still need them?

ive installed it using 3.11.7
it seems to have installed the packages correctly
but this erro still occurs when trying to change widget geometry
  File "C:\Users\58147793\AppData\Local\Programs\Python\Python312\Lib\tkinter\__init__.py", line 1962, in __call__
    return self.func(*args)
           ^^^^^^^^^^^^^^^^
  File "C:\Users\58147793\AppData\Local\Programs\Python\Python312\Lib\site-packages\customtkinter\windows\widgets\ctk_button.py", line 554, in _clicked
    self._command()
  File "C:\Users\58147793\ctk_theme_builder\lib\control_panel_v.py", line 948, in <lambda>
    command=lambda: self.launch_widget_geometry('CTkOptionMenu'))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\58147793\ctk_theme_builder\lib\control_panel_v.py", line 1054, in launch_widget_geometry
    geometry_dialog = GeometryDialog(master=self, widget_type=widget_type,
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\58147793\ctk_theme_builder\lib\geometry_dialog_v.py", line 25, in __init__
    self.enable_tooltips = mod.preference_setting(db_file_path=DB_FILE_PATH, scope='user_preference',
                           ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'lib.ctk_theme_builder_m' has no attribute 'preference_setting'```

ive installed it using 3.11.7 it seems to have installed the packages correctly but this error still occurs when trying to change widget geometry
  File "C:\Users\58147793\AppData\Local\Programs\Python\Python312\Lib\tkinter\__init__.py", line 1962, in __call__
    return self.func(*args)
           ^^^^^^^^^^^^^^^^
  File "C:\Users\58147793\AppData\Local\Programs\Python\Python312\Lib\site-packages\customtkinter\windows\widgets\ctk_button.py", line 554, in _clicked
    self._command()
  File "C:\Users\58147793\ctk_theme_builder\lib\control_panel_v.py", line 948, in <lambda>
    command=lambda: self.launch_widget_geometry('CTkOptionMenu'))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\58147793\ctk_theme_builder\lib\control_panel_v.py", line 1054, in launch_widget_geometry
    geometry_dialog = GeometryDialog(master=self, widget_type=widget_type,
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\58147793\ctk_theme_builder\lib\geometry_dialog_v.py", line 25, in __init__
    self.enable_tooltips = mod.preference_setting(db_file_path=DB_FILE_PATH, scope='user_preference',
                           ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'lib.ctk_theme_builder_m' has no attribute 'preference_setting'```

Please double check the Python version using:
venv\scripts\activate.bat
python --version

If you look at the path: ">   File "C:\Users\58147793\AppData\Local\Programs\Python\Python312\Lib\tkinter\__init__.py", line 1962, in __call__"
It references Python 3.12.

Did you use the Beta version, 3.0.0, zip file that I supplied?
Did you remove the previous install location (the ctk_theme_builder directory) before retrying?

PS C:\Users\58147793\ctk_theme_builder> .\venv\scripts\activate
(venv) PS C:\Users\58147793\ctk_theme_builder> python --version
Python 3.12.1
(venv) PS C:\Users\58147793\ctk_theme_builder>

i used 3.0.0 and i removed the old ctk_theme_builder directory

i run theme_builder_setup.py with this command
 python3117 theme_builder_setup.py -p "C:\Users\58147793\Downloads\ctk_theme_builder-3.0.0.zip"
as i said here the command python uses python 3.12.1 and python3117 uses 3.11.7

logs : log.zip
theme_builder_setup.py output:
theme_builder_setup.txt
ctk_theme_builder_output:
Traceback (most recent call last):
  File "C:\Users\58147793\AppData\Local\Programs\Python\Python312\Lib\tkinter\__init__.py", line 1962, in __call__
    return self.func(*args)
           ^^^^^^^^^^^^^^^^
  File "C:\Users\58147793\ctk_theme_builder\venv\Lib\site-packages\customtkinter\windows\widgets\ctk_button.py", line 554, in _clicked
    self._command()
  File "C:\Users\58147793\ctk_theme_builder\lib\control_panel_v.py", line 869, in <lambda>
    command=lambda: self.launch_widget_geometry('CTkButton'))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\58147793\ctk_theme_builder\lib\control_panel_v.py", line 1054, in launch_widget_geometry
    geometry_dialog = GeometryDialog(master=self, widget_type=widget_type,
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\58147793\ctk_theme_builder\lib\geometry_dialog_v.py", line 25, in __init__
    self.enable_tooltips = mod.preference_setting(db_file_path=DB_FILE_PATH, scope='user_preference',
                           ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'lib.ctk_theme_builder_m' has no attribute 'preference_setting'

i run theme_builder_setup.py with this command  python3117 theme_builder_setup.py -p "C:\Users\58147793\Downloads\ctk_theme_builder-3.0.0.zip" as i said here the command python uses python 3.12.1 and python3117 uses 3.11.7

If you have removed 3.12 Python,  something very odd is happening. It appears that it is still, somehow, picking up Python 3.12. The theme_builder_setup.py, makes an OS call to a execute a BAT file which also calls Python to create a virtual  environment, it is that which appears to be picking up Python 3.12, so I am guessing that is still sitting around, and is on your Windows %PATH%, appearing before the entry for Python 3.11.
All this is academic, to some extent. I installed Python 3.12 on my laptop tonight, and used the 3.0.0 zip to try an install. Again it went fine. I am starting to think that something has gone awry on your computer. I'll take a look at the logs, but I am doubtful of a resolution, without direct hands-on.

logs : log.zip theme_builder_setup.py output: theme_builder_setup.txt ctk_theme_builder_output:
Traceback (most recent call last):
  File "C:\Users\58147793\AppData\Local\Programs\Python\Python312\Lib\tkinter\__init__.py", line 1962, in __call__
    return self.func(*args)
           ^^^^^^^^^^^^^^^^
  File "C:\Users\58147793\ctk_theme_builder\venv\Lib\site-packages\customtkinter\windows\widgets\ctk_button.py", line 554, in _clicked
    self._command()
  File "C:\Users\58147793\ctk_theme_builder\lib\control_panel_v.py", line 869, in <lambda>
    command=lambda: self.launch_widget_geometry('CTkButton'))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\58147793\ctk_theme_builder\lib\control_panel_v.py", line 1054, in launch_widget_geometry
    geometry_dialog = GeometryDialog(master=self, widget_type=widget_type,
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\58147793\ctk_theme_builder\lib\geometry_dialog_v.py", line 25, in __init__
    self.enable_tooltips = mod.preference_setting(db_file_path=DB_FILE_PATH, scope='user_preference',
                           ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'lib.ctk_theme_builder_m' has no attribute 'preference_setting'

Strange, I can't see any problems in the logs. Does it still error with a timeout?

If you have removed 3.12 Python, something very odd is happening. It appears that it is still, somehow, picking up Python 3.12. The theme_builder_setup.py, makes an OS call to a execute a BAT file which also calls Python to create a virtual environment, it is that which appears to be picking up Python 3.12, so I am guessing that is still sitting around, and is on your Windows %PATH%, appearing before the entry for Python 3.11.
All this is academic, to some extent. I installed Python 3.12 on my laptop tonight, and used the 3.0.0 zip to try an install. Again it went fine. I am starting to think that something has gone awry on your computer. I'll take a look at the logs, but I am doubtful of a resolution, without direct hands-on.

I havnt tried removing 3.12 yet thats the next thing i will try

Strange, I can't see any problems in the logs. Does it still error with a timeout?

No and this problem doesnt crash the program only (what i presume to be) the widget geometry editor

so i removed python 3.12.1 from %path% and added 3.11.7 so now the  python command will use 3.11.7
... the venv version is still 3.12.1

i have deleted 3.12.1 completely (its stil in the recycle bin) and it doesnt create a venv at all
theme_builder_setup_2024-02-22_10-25-02.log

Strange, I can't see any problems in the logs. Does it still error with a timeout?

No and this problem doesnt crash the program only (what i presume to be) the widget geometry editor Al the other problems i was heving are fixed

I have reproduced your problem - leave it with me.

I think I have a fix - I'll get it published asap.
Thanks for your patience on this.

Hi @PimWagemans09. Justto let you know, these should now be fixed with. Please see CTk Theme Builder Release 3.0.1 .
Thanks for you persistence and patience, in getting to the bottom of these - much appreciated!

im glad i could help!
the issues are indeed gone now
i gues this issue can be closed now


add katacoda embed(s) to advo docs

We need to get this in by Wed pm / Th am, latest, so we can test it before the demo on Fri

Bit tight on space: https://edb-docs-adv-josh-skele-suffgu.herokuapp.com/getting-started/connecting_to_postgres/python/02_django
Gonna see what I can do about that in the AM - might be able to use a different layout.

@matt-edb @cero-miedo take a look at the space issue that katacoda embeds present. Ideas? Something like a sidenav toggle? Or we could pop katacoda embeds in a modal over the page?

Let's try a modal. I'll mock up in the wireframe.

Alternatively, we could simply change the bootstrap container size from 1140 to 1380 (or similar)? Stripe doc's main content area is 1080 pixels, ours is currently ~840 (with some interior padding too). Personally I think this would be preferable. It also helps with the reduced content width we have when the table of contents is present. Thoughts?

@epbarger @matt-edb I like that idea, Evan. Matt, thoughts?

Awesome, let's do it!

I was working up a modal approach, but that's way better.

Calling this done via https://github.com/rocketinsights/edb_docs/pull/70 and https://github.com/rocketinsights/edb_docs/pull/67 - but we'll likely keep tweaking. Also some things I can do with scenario composition to help here (stick to shorter line lengths in code for instance)


BrokenLinkCallback: Ownership and Send + Sync?

I was looking around the codebase and noticed that Rust has determined that Parser was !Send and !Sync. I thought it was odd because from a very high level perspective, it seems reasonable to expect us to be able to send a parser object between threads without difficulty--synchronous access maybe not so much (I can't really imagine a use case for when a client might want to have a parser be access from multiple threads at the same time) but Send implies Sync so ¯\_(ツ)_/¯.
It looks like the primary problem is that the type BrokenLinkCallback isn't Send nor Sync because we don't require the trait object to be send or sync:
pub type BrokenLinkCallback<'a> =
    Option<&'a mut dyn FnMut(BrokenLink) -> Option<(CowStr<'a>, CowStr<'a>)>>;

A quick fix is to just require the trait object reference to be Send + Sync:
pub type BrokenLinkCallback<'a> =
    Option<&'a mut (dyn Send + Sync + FnMut(BrokenLink) -> Option<(CowStr<'a>, CowStr<'a>)>)>;

but since BrokenLinkCallback is part of the public API, this would be a breaking change as it restricts what functions are accepted.
As a refresher on Closure auto-impl rules:

A closure is Sync if all captured variables are Sync.
A closure is Send if all variables captured by non-unique immutable reference are Sync, and all values captured by unique immutable or mutable reference, copy, or move are Send.

I might be wrong, but I don't think adding this requirement would be too difficult on users. I would expect most use cases to already be Send and Sync (our tests need no modification, for example), and I believe the current definition &mut dyn FnMut already forces some sort of Send/Sync primitive around the Parser itself, if they need to do so.
On that note, is there a particular reason why we accept a &mut to a callback, instead of an owned instance we Box instead or have Parser be generic over the trait?

It would have been good to have this, but I am afraid that this would be too restrictive. We could no longer capture any mutable references, since they aren't Sync. This would effectively limit the callbacks to Fn(BrokenLink) -> ....

On that note, is there a particular reason why we accept a &mut to a callback, instead of an owned instance we Box instead or have Parser be generic over the trait?

 Boxing obviously has an allocation cost, and using static dispatch could be faster. However, there are costs for static dispatch too. There's increased compilation time, increased binary size (when using different callbacks) and most importantly the possible inlining of very cold code. That last point is also the reason why we didn't sweat too much over this decision: the callback should be called very infrequently and hence the design shouldn't impact performance very much at all.

I appreciate the thought that's gone into the design of pulldown-cmark's API, in particular around the callback as it sounds like a difficult set of tradeoffs to deal with. But I do have a different perspective on one point:

the callback should be called very infrequently and hence the design shouldn't impact performance very much at all.

This is not necessarily a safe assumption to make for all users of the library. For instance, in a wiki system I've built, all links are written as [Thing], and the broken link callback does the work to both resolve that link (as /pages/thing/, for instance), and append the current page to a list of "backlinks" for the "Thing" page (requiring a mutable reference to a hash table). The broken link callback is definitely on the typical path for the program I've written.


Valid Flow fails

According to the website Flow is supported. But this example:
// @flow
const planets = {
  earth: 12756,
  mars: 6792
}
type Planet = $Keys<typeof planets>
const planet: Planet = 'mars'
return planet

fails with:
$ npx standard --parser babel-eslint --plugin flowtype minimal.js
standard: Use JavaScript Standard Style (https://standardjs.com)
  /home/vmx/tmp/eslintflow/minimal.js:6:15: '$Keys' is not defined.

The rule flowtype/define-flow-type would solve the problem. But if you would add this rule to eslint-config-standard'a eslintrc.json it would error if you run standard withoutt the --plugin flowtype.
Is there a way to resolve this problem that would be accepted? I'd be happy to work on a PR.

@vmx It's intentionally not easy to enable additional rules in standard, though in certain situations like yours, this can cause some trouble.
In the meantime, I recommend using standardx which provides an escape-hatch of sorts for these types of situations. It allows you enable additional rules.
Eventually, I'd like to solve this problem more elegantly in a future version of standard.


Multilang: Fix text overlap in pt-BR translation

something in my PC environment had distorted the UI aspect ratio and didn't demonstrate the overlap of texts. Now I double-checked on comma3 itself

Before:

make sure to rebase, think you're based on an old commit

make sure to rebase, think you're based on an old commit

Yes, you got it right, now I've synced with master and re checked again, thanks!


Added all components for Border Radius Inline editor

Added all components for inlineBorderRadiusEditor

Hey, awesome - this is super cool @feihaozi77 👍 Some thoughts...

When moving the slider, is it possible to change the value when the slider is moving and not just when the slider stops moving?

I think the oninput event should do the trick
onchange only fires at the end

Let's not have any decimal spots in the pixel values

Instead of 12.75px let's just use 13px
We can set a number of fixed steps for the input range element

@flukeout Sure I'll work on the parts you mentioned.

@flukeout made some changes

Thanks for your patience @feihaozi77 - reviewing again now.

This is getting really good! Here is some more feedback...

When adjusting the top-left-corner, is it possible to not also change the all-corners value?

Is it possible to retain the values of the individual corners so they don't reset when you switch to the All corners mode?

I'm imagining a way where the individual and all modes have their own separate "memory". So if you make changes in one, it doesn't affect the other. So, let's say..

I make some changes to individual corners
I switch to all corners mode
The individual corner value sliders are not affected
If i switch back to individual corners mode, we apply those stored values to the CSS

Does it make sense? Let me know! Thanks so far!!

@flukeout I made the changes you need to seperate the allCornerSlider and the individualSlider, please review when you are avaliable

Awesome, will try it out now.

OK, great! I think that works much better. Some more requests:
Request 1
Is it possible to show the purple editor icon when there isn't a value yet for the CSS rule? I'm thinking of the case when the user has typed..
border-radius:  but hasn't started to add any values yet.
I think it would be good to show the icon right then so they could start using the UI right away if they wanted to.
Request 2
Is it possible to show the pixel value after the range slider? Like this...
all corners ==0========== 12px
Once we get these changes in I can make a PR against your branch with styling updates, etc.
👍

@flukeout let me clarify request1, do you mean you want the purple icon to pop up automatically without user clicking the text when user finished typing something like border-radius:

@flukeout I think I'll get your request 2 done by tomorrow but the 1st request seems pretty hard, I've been stucking there for awhile, I looked into the PopoverWidget.js file, the positionPopover function is being called, if I set a break point inside the positionPopover function, the bubble was showed at first when the execution was stopping at the break point, but once I step over the break point the bubble just dispeared. I could not figure out this, maybe some one who are more familliar with codemirror could help

Hey @feihaozi77 - yeah with regard to Request 1 - that's what I had in mind. If it's proving too time consuming, let's not worry about it too much at this point. We could always try to solve it with another effort down the road.

@flukeout request2 completed

Awesome @feihaozi77 - will take a look a little later today!

Looking now.

This is great @feihaozi77! I've got a question: is it possible for the UI to be aware of the units used in the rule? For example, if I have border-radius: 5px 10%; and open the UI, it converts all of the units to px. Is it possible to keep %?
Then, on top of that, maybe we should add a toggle to each value that lets the user choose then unit. Something like this...
top-left-corner ===0======= 10 [px / % / em]
Let me know if that is possible. Thanks!

@flukeout  regarding your request "if I have border-radius: 5px 10%; and open the UI, it converts all of the units to px. Is it possible to keep %?",   should be able to do that. not sure about the toggle part yet.

@flukeout Added the part for keeping %, em and px units

Testing now @feihaozi77 - thanks!

Hey @feihaozi77, when opening a single-value border-radius, is it possible to also populate that value into all of the corner values? See the screencap below, when I open it, all of the values are set to 0...

@flukeout sure, I will work on that,  and right now i'm trying to add the radio buttons for each of the corners to toggle their units

Ooops, sorry accidentally closed and opened. Awesome, thanks for the update, looking forward to it!

@humphd @flukeout radio buttons for unit toggling are added and regarding your last comment ,
individual corners are now syncronized with allCorner slider.

Testing again @feihaozi77 - thanks!

Hey awesome, it's working quite well. One issue that I have found is that when you close the UI, the cursor goes to the beginning of the page...

I'm also starting work on a PR to style the UI you made now. I'll probably be making changes to the markup and stylesheet, so if you could stop making changes there for a bit, that would be great—my PR will be easy to merge when it's ready. Does that sound good @feihaozi77

@flukeout That sounds great, I noticed the cursor issue you mentioned, and I had been working on that for a while but still no luck yet. Anywayz, I'll just stop here and you can work on the style

@feihaozi77 Making some progress, here's how I have it looking so far. I'll create a PR against your master branch (where your work is happening) at the end of my day here in a couple of hours. That way you can merge it and continue working and I can pick it up again tomorrow.

@flukeout wow that looks so pretty!!!

Haha, thanks @feihaozi77. I'm going to make my PR now so you can check it out. In my PR I've removed the top right and top left labels, etc to try to save space. I'm not sure if that's a good idea yet. I'm worried if the editor is too narrow, the UI wont' have enough room.

Okay @feihaozi77 - I've filed the PR - https://github.com/feihaozi77/brackets/pull/1

@feihaozi77 - testing again now, thanks!

I'm going to make some final styling changes and then we'll get a code review from either @gideonthomas or @humphd

Ping @humphd—ready for review!

@humphd ready for code review!

@humphd made some changes about the coding style based on your request, please review

@humphd ready for review again

For localization see https://github.com/mozilla/thimble.mozilla.org/wiki/Localization and https://github.com/mozilla/brackets/wiki/Localizing-Bramble

@humphd @gideonthomas I made a Pull Request to the thimble repo. this PR was about adding the localized strings to the locales folder. Please take a look and let me know if that's OK. Here is the PR link https://github.com/mozilla/thimble.mozilla.org/pull/2585

@humphd @gideonthomas ready for review!


Library and Angular CLI app in a monorepo (Module not found: import paths cannot be resolved)

Type of Issue

[x] Bug Report
[ ] Feature Request

Description
After trying it on a dead simple app and failed, i cloned the example repo, ng-packaged, and got the same exact error. It builds without a problem, but the moment you try to import the lib to the app you get the error
ERROR in ./src/app/app.module.ts
Module not found: Error: Can't resolve '@my/lib' in 'D:\Work\testing\Angular2\ng-packaged\src\app'
resolve '@my/lib' in 'D:\Work\testing\Angular2\ng-packaged\src\app'
  Parsed request is a module
  using description file: D:\Work\testing\Angular2\ng-packaged\package.json (relative path: ./src/app)
    Field 'browser' doesn't contain a valid alias configuration
  after using description file: D:\Work\testing\Angular2\ng-packaged\package.json (relative path: ./src/app)
    resolve as module
      D:\Work\testing\Angular2\ng-packaged\src\app\node_modules doesn't exist or is not a directory
      D:\Work\testing\Angular2\ng-packaged\src\node_modules doesn't exist or is not a directory
      D:\Work\testing\Angular2\node_modules doesn't exist or is not a directory
      D:\Work\node_modules doesn't exist or is not a directory
      D:\node_modules doesn't exist or is not a directory
      looking for modules in D:\Work\testing\Angular2\ng-packaged\node_modules
        using description file: D:\Work\testing\Angular2\ng-packaged\package.json (relative path: ./node_modules)
          Field 'browser' doesn't contain a valid alias configuration
        after using description file: D:\Work\testing\Angular2\ng-packaged\package.json (relative path: ./node_modules)
          using description file: D:\Work\testing\Angular2\ng-packaged\package.json (relative path: ./node_modules/@my/lib)
            no extension
              Field 'browser' doesn't contain a valid alias configuration
              D:\Work\testing\Angular2\ng-packaged\node_modules\@my\lib doesn't exist
            .ts
              Field 'browser' doesn't contain a valid alias configuration
<EMAIL_ADDRESS>doesn't exist
            .js
              Field 'browser' doesn't contain a valid alias configuration
<EMAIL_ADDRESS>doesn't exist
            as directory
              D:\Work\testing\Angular2\ng-packaged\node_modules\@my\lib doesn't exist
      looking for modules in D:\Work\testing\node_modules
        No description file found
        Field 'browser' doesn't contain a valid alias configuration
        No description file found
        no extension
          Field 'browser' doesn't contain a valid alias configuration
          D:\Work\testing\node_modules\@my\lib doesn't exist
        .ts
          Field 'browser' doesn't contain a valid alias configuration
<EMAIL_ADDRESS>doesn't exist
        .js
          Field 'browser' doesn't contain a valid alias configuration
<EMAIL_ADDRESS>doesn't exist
        as directory
          D:\Work\testing\node_modules\@my\lib doesn't exist
      looking for modules in D:\Work\testing\Angular2\ng-packaged\node_modules
        using description file: D:\Work\testing\Angular2\ng-packaged\package.json (relative path: ./node_modules)
          Field 'browser' doesn't contain a valid alias configuration
        after using description file: D:\Work\testing\Angular2\ng-packaged\package.json (relative path: ./node_modules)
          using description file: D:\Work\testing\Angular2\ng-packaged\package.json (relative path: ./node_modules/@my/lib)
            no extension
              Field 'browser' doesn't contain a valid alias configuration
              D:\Work\testing\Angular2\ng-packaged\node_modules\@my\lib doesn't exist
            .ts
              Field 'browser' doesn't contain a valid alias configuration
<EMAIL_ADDRESS>doesn't exist
            .js
              Field 'browser' doesn't contain a valid alias configuration
<EMAIL_ADDRESS>doesn't exist
            as directory
              D:\Work\testing\Angular2\ng-packaged\node_modules\@my\lib doesn't exist
      looking for modules in D:\Work\testing\Angular2\ng-packaged\src
        using description file: D:\Work\testing\Angular2\ng-packaged\package.json (relative path: ./src)
          Field 'browser' doesn't contain a valid alias configuration
        after using description file: D:\Work\testing\Angular2\ng-packaged\package.json (relative path: ./src)
          using description file: D:\Work\testing\Angular2\ng-packaged\package.json (relative path: ./src/@my/lib)
            no extension
              Field 'browser' doesn't contain a valid alias configuration
              D:\Work\testing\Angular2\ng-packaged\src\@my\lib doesn't exist
            .ts
              Field 'browser' doesn't contain a valid alias configuration
<EMAIL_ADDRESS>doesn't exist
            .js
              Field 'browser' doesn't contain a valid alias configuration
<EMAIL_ADDRESS>doesn't exist
            as directory
              D:\Work\testing\Angular2\ng-packaged\src\@my\lib doesn't exist
      looking for modules in D:\Work\testing\Angular2\ng-packaged\src
        using description file: D:\Work\testing\Angular2\ng-packaged\package.json (relative path: ./src)
          Field 'browser' doesn't contain a valid alias configuration
        after using description file: D:\Work\testing\Angular2\ng-packaged\package.json (relative path: ./src)
          using description file: D:\Work\testing\Angular2\ng-packaged\package.json (relative path: ./src/@my/lib)
            no extension
              Field 'browser' doesn't contain a valid alias configuration
              D:\Work\testing\Angular2\ng-packaged\src\@my\lib doesn't exist
            .ts
              Field 'browser' doesn't contain a valid alias configuration
<EMAIL_ADDRESS>doesn't exist
            .js
              Field 'browser' doesn't contain a valid alias configuration
<EMAIL_ADDRESS>doesn't exist
            as directory
              D:\Work\testing\Angular2\ng-packaged\src\@my\lib doesn't exist

The angular app.module.ts
import { BrowserModule } from '@angular/platform-browser';
import { NgModule } from '@angular/core';
import { FormsModule } from '@angular/forms';
import { HttpModule } from '@angular/http';

import { AppComponent } from './app.component';
import { MyLibModule } from '@my/lib';

@NgModule({
  declarations: [
    AppComponent
  ],
  imports: [
    BrowserModule,
    FormsModule,
    HttpModule,
    MyLibModule.forRoot()
  ],
  providers: [],
  bootstrap: [AppComponent]
})
export class AppModule { }

tsconfig.app.json
{
  "extends": "../tsconfig.json",
  "angularCompilerOptions": {
    "paths": {
      "@my/lib": [ "../dist/my-lib" ]
    }
  },
  "compilerOptions": {
    "outDir": "../out-tsc/app",
    "module": "es2015",
    "baseUrl": "",
    "types": [],
    "paths": {
      "@my/lib": [ "../dist/my-lib" ]
    }
  },
  "exclude": [
    "test.ts",
    "**/*.spec.ts"
  ]
}

main tsconfig.json
{
  "compileOnSave": false,
  "compilerOptions": {
    "outDir": "./dist/out-tsc",
    "baseUrl": "src",
    "sourceMap": true,
    "declaration": false,
    "moduleResolution": "node",
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "target": "es5",
    "typeRoots": [
      "node_modules/@types"
    ],
    "lib": [
      "es2016",
      "dom"
    ],
    "paths": {
      "@my/lib": [ "dist/my-lib" ]
    }
  },
  "exclude": [
    ".ng_build"
  ]
}

How To Reproduce
$ yarn install
$ yarn build:lib
$ ng serve
Version Information
ng-packagr: v1.0.1
node: v8.1.2
@angular: v4.3.2
rxjs:
zone.js:

Hi @mbouclas,
there is an issue with the paths mapping in the tsconfig. Builds started failing with angular cli 1.3.x - issue angular-cli#7341
You can try to use a symlink instead. Something like:
node_modules/@my/lib --symlink to--> dist/@my/lib
In another project, a workaround is to copy the dist folder to the src folder angular cli, i.e.:
src
 |- app
 |- @my
     |- lib
 |- tsconfig.app.json (paths mapping to "./@my/lib")

That somehow works w/ angular cli 1.3.2.

got it, thanks.

@dherges when creating the symlink to node_modules it does seem to import it, but it produces the following error
Uncaught Error: Unexpected value 'MyModule' imported by the module 'AppModule'. Please add a @NgModule annotation.

Sure enough the exported module looks like this
export declare const COMPONENTS: any[];

export declare class MyModule {
    constructor();
    static forRoot(): {
        ngModule: typeof MyModule ;
        providers: any[];
    };
}

So, using the symlink trick, do we still have to add something to the app tsconfig or am i missing something?

No, with the symlink you shuld remove paths from tsconfig. So either tsconfig or symlink. Do not use both in parallel (don't know what happens then...)
Is the symlink in the correct direction? I always happen to exchange source and target. It needs to be like this:
|- dist
    |- @my
        |- lib
            |- package.json
|- node_modules
   |- @my
       |- lib

Now, when you do cat<EMAIL_ADDRESS>the file from the dist folder should show up. I just want to make sure no silly mistakes are in place :)
Or:
$ ls -al node_modules/@my
lib --> ../../dist/@my/lib

So the command for ln would/should be something like:
ln -sf ../../dist/@my/lib node_modules/@my/lib

Yes the symlink is correct and i do see the changes happen. Also, i have removed everything path related from tsconfig. The error produced suggests that it did find the package in node_modules, it just doesn't like  the way it's formatted causes it complains for a missing annotation @NgModule
Is there a chance that it is loading the wrong file? Also. noticed that this message shows up when importing the module plain, if i try to import it like MyModule.forRoot() i get another error
vendor.bundle.js:29253 Uncaught Error: Unexpected value '[object Object]' imported by the module 'AppModule'. Please add a @NgModule annotation.

For anyone with a similar issue, my way of solving this was not to symlink (tested on windows only using both soft and hard links). For some reason this just won't work. What does work though is copying the dist output to your node_modules folder.
I ended up using a chokidar watcher which builds the package and then copies the dist to node_modules.
@dherges thanks for your time

@mbouclas your solution worked for me as well after a failed attempt at running a yarn link or npm link. Thanks for the heads up on this.

I'm getting the same type of error even not using "paths".
I'm trying to create a library from a module inside a regular angular-cli project.
I followed the guide here: https://medium.com/@ngl817/building-an-angular-4-component-library-with-the-angular-cli-and-ng-packagr-53b2ade0701e but I'm getting the error reported in this issue.
You can find my test in this github project: https://github.com/riccardomariani/odata-v4-ng.
Any suggestion to solve the problem?
Thanks a lot in advance

Can you take a look at copying the dist artefacts and mapping paths to sources for a dev build
These solve the issues for TypeScript module resolution that came with angular/angular-cli#7341 - see https://github.com/angular/angular-cli/issues/7341#issuecomment-324839657
Regarding Unexpected value '[object Object]' imported ... Please add a @NgModule annotation., may have another cause. Please check and/or try to debug whether the import is properly resolved!

Actually the problem was caused by how I was importing classes from the library.
Based on the generated library code, Visual Studio Code proposed
import { ODataModule } from 'odata-v4-ng/odata-v4-ng'; or
import { ODataModule } from 'odata-v4-ng/public_api';
instead of
import { ODataModule } from 'odata-v4-ng';
Changing the name of the file generated by ng-packagr from odata-v4-ng.d.ts to index.d.ts solved the problem and now Visual Studio Code is proposing the right import.
So my question is: is this a misconfiguration of ng_packagr in my library? or is this the expected behavior?

I changed ng-package.json config file adding:     "flatModuleFile": "index"
and this solved the problem.
{
"$schema": "./node_modules/ng-packagr/ng-package.schema.json",
"lib": {
"entryFile": "public_api.ts",
"flatModuleFile": "index"
}
}

There are several ways to work with a library (and it's dist artefacts) and Angular ng CLI in the same repo:

copying dist artefacts: demonstrated by build:app in ng-packaged demo
using tsconfig paths mapping: demonstrated by build:dev in ng-packaged demo
symlink the dist folder (also yarn link for creating/managing the symlink), as in the in the consumer integration test.

@dherges I have tried all of these options, but the slight difference is that I intend on ultimately publishing my updated library to npm, so I'm trying to use npm link to replace the version in node_modules, and I do not want it to be a monorepo with the library in src.
My error is very similar to OP of this issue, but I have tried copying the dist artifacts (from a different project) to node_modules (with no path mapping) and I've tried doing the tsconfig path mapping with npm link and ng build --aot with --preserve-symlinks. They all seem to produce the same error, but only with AoT builds.
As others have stated, ng serve works fine.

This may be relevant https://github.com/angular/angular-cli/issues/4874


keyExists, getInt and getRaw fail when value to get is equal to zero

I'm trying to validate a form using Inspekt. One of the fields I'm trying to validate is a select box with values such as -1, 0, 1, 2, etc. I have set -1 as my invalid case, and all other cases are valid. However, when trying to get the value of the select, it works for any value except 0, which makes all of the above mentioned methods to fail (i.e.: return false).
I found it weird, so I thought I would mention it here. Any ideas?

FYI I've added tests, and in 0.6.0 it seems to work:
$inputarray['zeroint'] = 0;
// ............................
$this->assertSame(0, $this->cage->getRaw('zeroint'));


Branches with special characters break .dockstore.yml sync

Describe the bug
A .dockstore.yml workflow will not update on Dockstore (and by extension Terra) any branches that contain the following characters: (){}<>'Ó (and possibly others)
Context
I have a branch of analysis_pipeline_wdl that has a bugfix involving chrX. I got a little too clever for my own good and gave the branch a punny name (calling it a pun is a stretch, really). That being said, the name has gotten compliments and helped me find this bug, which I take to mean I should name all of my branches with puns going forward.
To Reproduce
Steps to reproduce the behavior:

Have any valid .dockstore.yml repo
Create a new branch with ( in the title, or whatever other special character you're in the mood to test
Go to Dockstore's github app logs and see the failure
Confirm that adding brand new branches with no special characters doesn't error and does update successfully (ie not just a temporary hiccup)

Expected behavior
For Dockstore to be truly international I'd prefer the first option, but there's four possible routes we could take this:

Support all characters that GitHub allows in branch names
Support parenthesis, apostrophes, CJK characters, and accented roman characters; and document which characters cannot be used
Support all ASCII extended characters, which includes accented roman characters
Support basic ASCII characters such as apostrophes -- we don't need to support Little Bobby Tables, but I'd appreciate it if we at least support me

Screenshots
Problematic branch in question plus the error itself

General testing (aaaaaaa did work)

💔

Desktop (please complete the following information):
Webservice - 1.12.0
UI - 2.9
Compose Setup - 1.12.0-rc.1
Deploy - 1.12.3
galaxyParsingLambdaVersion: 0.0.7
┆Issue is synchronized with this Jira Story
┆Fix Versions: Dockstore 1.14
┆Issue Number: DOCK-2195
┆Sprint: 106 - Don River
┆Issue Type: Story

@denis-yuen Do you think it is worth documenting this limitation in the meantime?

Some of these should work barring restrictions in https://git-scm.com/docs/git-check-ref-format

In qa, added, then deleted stop-drop\(ing-chr-X branch, and it properly showed up, then went away.
[Domain] - https://qa.dockstore.org
Webservice - fb9ebdf
UI - 2.10.1-107-ga76609e6
Compose Setup - 6f68328
Deploy - 1.13.2-31-g140c057
cwlParsingLambdaVersion: n/a
wdlParsingLambdaVersion: n/a
nextflowParsingLambdaVersion: n/a
galaxyParsingLambdaVersion: 0.0.6
checkUrlLambdaVersion: n/a


Examples as a standalone repository

It's tough copying and pasting code from Gomix or your ready snippets trying to piece them together as something that runs locally. It would be nice if there was another repository under hyperapp for examples.

Yes. https://github.com/hyperapp/hyperapp/issues/41 comes to mind as well.

@rbiggs Are you familiar with Gomix? There's tons of examples there and they are downloadable or exportable to GitHub.

Hmmm... I'm on Gomix but I don't see any option to download your examples. At the same time, seems like Hyperapp is still going through a lot of churn with its API, making it hard to keep separate examples updated. That said, it is brainless easy to download your Codepen examples.

Hyperapp is still going through a lot of churn with its API

Internally, yes, but the API is hopefully locked by now. There was an incident recently wrt to camelCase event names, but other than that we're not going to see any major changes. I'll release 1.0.0 when the book is ready and more examples are available, perhaps a live site or two.
For the record, in gomix, go to Project info and options, then select Advanced Options.

All I see is this

Any way, I did a copy/past from here to local versions. Everything runs local with Webpack building, etc. The only issue I've had was getting setLocation to work without Browserify. But then, you removed that example from GoMix.

@rbiggs Oh, okay, I see what's going on. You'd think Gomix would allow you to download whatever you wanted, but it seems that's not how it works, instead you need to "Remix This", which is their equivalent of GitHub's fork.
Then, you'll be the administrator of your own copy and you'll be able to download/export.

Oh, I see. 🙄 But of course they don't mention that anywhere, so I was clicking on things around in a circle for a half hour. Googled, etc. In the end I got what I wanted by copy/pasting because I didn't what them running with Browserify. I made each example standalone with its own server instance and a Webpack config to create a bundle. That way it's easier to fiddle with each example, add other things, etc.

Status Update: We're not using Gomix Glitch to host code examples anymore, but there are many examples available on CodePen. I keep my own pens always up-to-date; can't speak for others. I still don't want to maintain code examples in this repository.


Botkit anywhere - help

Hi,
I used this this fantastic middleware for a Facebook Messenger bot.
Now I would like to connect it to an mobile app, how can I do? Where should I connect the middleware to the websocket?
Thank you

same place.

Thanks you for your reply.
Could you show me how to do it?
Is something like:
In app.js
if (process.env.USE_WEBSOCKET) {...}
Thanks

take the botkit-starter-web and copy watson initialization code and controller.middleware.receive.use(watsonMiddleware.receive); from your previous project.
This line of code connects middleware to botkit.

This issue has no reproducible steps. Please re-open this issue if it still occurs.


Zulip Updates + help center tweak

CZO discussion
Message (links work):

--
Help center commit:
Before: https://zulip.com/help/moderating-open-organizations#minimize-spam
After:

Looks great, marked to merge once CI passes. Thanks @alya!


Enable translations

[x] Enable Greek 🇬🇷
[x] Enable Italian 🇮🇹
[x] Enable Serbian 🇷🇸

Codecov Report

Merging #231 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #231   +/-   ##
=======================================
  Coverage   92.64%   92.64%
=======================================
  Files          92       92
  Lines        1088     1088
  Branches      165      165
=======================================
  Hits         1008     1008
  Misses         65       65
  Partials       15       15

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 46f070d...5bcba2f. Read the comment docs.


Disentangle hardcoding and templating

Following the discussion in #79, I realized that the templating of l_max (which leads to loop unrolling) and hardcoding can have different l_max thresholds. This would let us reap the benefits of loop unrolling also for l_max>6, although I haven't done any speed tests for the moment. An additional benefit is that this would remove the ugly duplication of code in templates.hpp, where we can now have a single loop which is fully static. (Here I'm assuming that our new SPERICART_LMAX_TEMPLATED would be so high that at that point loop unrolling wouldn't be possible anymore, so that we wouldn't realistically need to try enforcing unrolling for the first part of super-high ls and the compiler will choose for us at what point it stops unrolling.)
The only issue is template instantiation. I'm doing it with if statements at the moment (and SPERICART_LMAX_TEMPLATED=10) to avoid having to instantiate all the other template parameters, which would be a nightmare. Let me know what you think @ceriottm @Luthaf.

What do the benchmarks look like before/after this change?

Note that the Julia "templated" code does more than just unroll. The key point is that it is able to keep all work on the stack, wheres a "flexible" code would need to use memory from the heap. This is one of the more problematic performance pitfalls in Julia.

I wasn't seeing any performance increase until I used this compiler-specific keyword (__attribute__((always_inline))). What I think is going on is that here
https://github.com/lab-cosmo/sphericart/blob/4295669a9a885eadd2e5df4dc4fafb2a6c134893/sphericart/include/templates.hpp#L641-L661
we have a TEMPLATED_L that is being fed as a function parameter to an "inline" function. The issue is that inline is just a suggestion to the compiler, and it can be ignored. If it is ignored, then TEMPLATED_L is not known and the loops inside the second function won't be unrolled.

****************** DOUBLE PRECISION ******************
Call without derivatives (no hardcoding) took 98.8981 ± 0.605256 ns / sample
Call with derivatives (no hardcoding) took 337.11 ± 1.67443 ns / sample
Call with second derivatives (no hardcoding) took 1336.61 ± 2.80391 ns / sample

Sample without derivatives took 106.918 ± 1.36062 ns / sample
Sample with derivatives took 255.564 ± 6.15337 ns / sample
Sample with second derivatives took 853.075 ± 86.4209 ns / sample

Call without derivatives took 78.1945 ± 0.471599 ns / sample
Call with derivatives took 302.576 ± 1.69244 ns / sample
Call with second derivatives took 1197.27 ± 8.23424 ns / sample
Call without derivatives (normalized) took 87.2571 ± 0.521217 ns / sample
Call with derivatives (normalized) took 353.931 ± 7.2684 ns / sample
Call with second derivatives (normalized) took 2393.58 ± 3.87189 ns / sample

================ Low-l timings ===========
L=1 (no h-c) values              took 4.71103 ± 0.164448 ns / sample
L=1 (no h-c) values+derivatives  took 6.90779 ± 0.129 ns / sample
L=1 hardcoded values             took 1.60676 ± 0.0933306 ns / sample
L=1 hardcoded values+derivatives took 3.61443 ± 0.117717 ns / sample

L=2 (no h-c) values              took 8.635 ± 0.178832 ns / sample
L=2 (no h-c) values+derivatives  took 8.63524 ± 0.142311 ns / sample
L=2 hardcoded values             took 3.41483 ± 0.114826 ns / sample
L=2 hardcoded values+derivatives took 13.1237 ± 0.225533 ns / sample

L=3 (no h-c) values              took 12.7715 ± 0.283376 ns / sample
L=3 (no h-c) values+derivatives  took 27.6311 ± 0.305504 ns / sample
L=3 hardcoded values             took 11.9532 ± 0.204903 ns / sample
L=3 hardcoded values+derivatives took 21.4224 ± 0.232066 ns / sample

L=4 (no h-c) values              took 19.1465 ± 0.265671 ns / sample
L=4 (no h-c) values+derivatives  took 46.6761 ± 0.323387 ns / sample
L=4 hardcoded values             took 9.95259 ± 0.155389 ns / sample
L=4 hardcoded values+derivatives took 37.4713 ± 0.329868 ns / sample

L=5 (no h-c) values              took 22.1213 ± 0.256414 ns / sample
L=5 (no h-c) values+derivatives  took 70.7344 ± 0.658015 ns / sample
L=5 hardcoded values             took 15.3568 ± 0.2742 ns / sample
L=5 hardcoded values+derivatives took 56.6069 ± 0.547587 ns / sample

L=6 (no h-c) values              took 31.4157 ± 0.364123 ns / sample
L=6 (no h-c) values+derivatives  took 105.335 ± 1.11829 ns / sample
L=6 hardcoded values             took 22.308 ± 0.304872 ns / sample
L=6 hardcoded values+derivatives took 87.8669 ± 1.19881 ns / sample

And this is the main branch:
****************** DOUBLE PRECISION ******************
Call without derivatives (no hardcoding) took 104.425 ± 0.57164 ns / sample
Call with derivatives (no hardcoding) took 300.287 ± 1.87295 ns / sample
Call with second derivatives (no hardcoding) took 1342.6 ± 2.97078 ns / sample

Sample without derivatives took 109.738 ± 2.0463 ns / sample
Sample with derivatives took 266.269 ± 95.4806 ns / sample
Sample with second derivatives took 853.931 ± 77.1591 ns / sample

Call without derivatives took 80.3673 ± 0.451071 ns / sample
Call with derivatives took 330.482 ± 1.50251 ns / sample
Call with second derivatives took 1357.74 ± 13.4713 ns / sample
Call without derivatives (normalized) took 89.9167 ± 0.639336 ns / sample
Call with derivatives (normalized) took 379.961 ± 3.39652 ns / sample
Call with second derivatives (normalized) took 2438.88 ± 3.19248 ns / sample

================ Low-l timings ===========
L=1 (no h-c) values              took 3.76709 ± 0.116769 ns / sample
L=1 (no h-c) values+derivatives  took 8.30298 ± 0.169697 ns / sample
L=1 hardcoded values             took 1.61501 ± 0.321693 ns / sample
L=1 hardcoded values+derivatives took 3.60945 ± 0.156557 ns / sample

L=2 (no h-c) values              took 7.19116 ± 0.112577 ns / sample
L=2 (no h-c) values+derivatives  took 7.19082 ± 0.122308 ns / sample
L=2 hardcoded values             took 3.43054 ± 0.10358 ns / sample
L=2 hardcoded values+derivatives took 13.1471 ± 0.18961 ns / sample

L=3 (no h-c) values              took 11.9002 ± 0.190871 ns / sample
L=3 (no h-c) values+derivatives  took 29.3307 ± 0.287947 ns / sample
L=3 hardcoded values             took 11.9445 ± 0.189964 ns / sample
L=3 hardcoded values+derivatives took 21.4568 ± 0.212694 ns / sample

L=4 (no h-c) values              took 18.6302 ± 0.208289 ns / sample
L=4 (no h-c) values+derivatives  took 47.7342 ± 0.364463 ns / sample
L=4 hardcoded values             took 9.95611 ± 0.134674 ns / sample
L=4 hardcoded values+derivatives took 38.019 ± 0.309817 ns / sample

L=5 (no h-c) values              took 27.5422 ± 0.323073 ns / sample
L=5 (no h-c) values+derivatives  took 69.7555 ± 0.565318 ns / sample
L=5 hardcoded values             took 15.2972 ± 0.1696 ns / sample
L=5 hardcoded values+derivatives took 56.8899 ± 0.594101 ns / sample

L=6 (no h-c) values              took 39.1725 ± 0.361854 ns / sample
L=6 (no h-c) values+derivatives  took 107.807 ± 1.31283 ns / sample
L=6 hardcoded values             took 22.193 ± 0.341033 ns / sample
L=6 hardcoded values+derivatives took 88.1554 ± 1.20667 ns / sample

It does not look like a big improvement on the benchmarks. Some cases are faster, other slower but not by much more than 5%.

The issue is that inline is just a suggestion to the compiler, and it can be ignored.

Yes, inline does NOT mean inline this function, and never had. Instead it says that the code for this function is provided inline (typically in a header), and one of the copies (whichever one) should be kept in the final binary. This is related to One Definition Rule (ODR).
If we want/need to use __attribute__((always_inline)), we'll have to check that the compiler supports it first in cmake. This is to ensure we can also support compilers that are not GCC (MSVC on Windows, the Intel compilers and the IBM compilers). We could also see if there is a standard C++17 attribute (looking like this [[<some_name>]]) instead.

You're right: the performance improvement doesn't seem to be there. I had a look and __attribute__((always_inline)) is very specific to gcc


support for adding empty directories

+1
Maybe take a hint from archiver where you specify an empty string as filename to add a empty directory.


'FileDataset' object has no attribute 'SliceLocation'

I have an rt-struct.dcm file by itself, I want to extract a 3D volumetric mask from this. From loading the Rt-struct using dicominfo on MATLAB I can see that it has no SliceLocation attribute. However, it does have ROI contour attributes. All I am trying to do is extract these ROI contours (which are currently stored in Cartesian coordinates) as a mask. Is it possible to do this with rt-utils?

@carluri is this something you want to be supporting? It was written under the assumption that we would always the actual DCM files associated with the RT-STUCTS as there is quite a bit of information you need from the DCM files to generate the RT struct.
If you wanted to change this this should point you in the right direction:
C:\Users\cadam\programming\rt-utils>git grep -n SliceLocation
rt_utils/image_helper.py:25:    series_data.sort(key=lambda ds: ds.SliceLocation, reverse=False)
rt_utils/image_helper.py:138:    z_indicies = np.ones((contour.shape[0], 1)) * series_slice.SliceLocation

I came here from @pchlap's comment in https://github.com/qurit/rt-utils/pull/21#issue-641906262. I vendored into PyMedPhys some code by @Fincap and @sjswerdloff that achieves this.
The code that does this sorting within PyMedPhys is over at:
https://github.com/pymedphys/pymedphys/blob/9b16b9cbc293276422e7209c658160962405fba9/lib/pymedphys/_dicom/sorting.py#L1-L57
The original code is over at:
https://github.com/didymo/OnkoDICOM/blob/cfab3aefb1427ab251a5de3df1b04d729ecd4b5d/src/Model/ImageLoading.py#L112-L130
And the relicensing discussion occurred over at:
https://github.com/pymedphys/pymedphys/pull/1458#discussion_r598630630

To add onto this @johndgiese over at innolitics also has a similar solution to this:
https://github.com/innolitics/dicom-numpy/blob/204e95594a527bbab1444f9248432ffa01af024c/dicom_numpy/combine_slices.py#L80-L88

Hi, I'm having the same issue rasterizing structures on some CT series that are missing the optional SliceLocation attribute. It's possible to sanitize our files in the short term, but in the future it would be great to rely on ImagePositionPatient.


⚠ Processors has degraded performance

In c248588, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 94 ms

Resolved: Processors performance has improved in 411518f.


Image compression on CC.org // Profile Pictures

Hello, I am trying to update the Profile Photo for Bilal (https://creativecommons.org/author/bilalranderee/) because the way it currently displays is low-res / low quality / blurry
But no matter how high quality of a version of the image I upload, it downgrades/compresses it so that it looks crummy
Can we change how much compression is added to these images? Or would someone help me by suggestion another way to get a decent looking image online for Bilal?
Thanks
cc @victoriaheath

Hello, I am outreachy applicant.I can work on this issue ?

This issue is not marked "help wanted" and is not suitable for external contribution.


Add custom class prefix to remark-prismjs

Prismjs's default classes for syntax highlighting can come into conflict with your own classes. Adding this plugin from prismjs lets us prefix its classnames so that we can avoid this.
Before:

After:

I've ran into this issue myself and you can see examples of this causing issues even in Gridsome starters like this one. Notice how the code block looks weird, with anomalies like the giant number 5? That's from the conflicting number class from Bulma and number in Prismjs. I wonder if there should even be a prefix by default, if this really is so common, but I have implemented this as an option.

Thanks, @BernardZhao. Seems like a useful option to me.


Fixing build issue on Node 6

Fixing an odd Node 6 issue in which ember cli complains about isProduction not being defined. I suspect this may have something to do with variable declaration order.

[Current coverage][cc-pull] is 87.52%

Merging [#153][cc-pull] into [master][cc-base-branch] will decrease coverage by 0.23%

2 files (not in diff) in app were modified. more

Misses +2
Hits -2

@@             master       #153   diff @@
==========================================
  Files            36         36
  Lines           761        761
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits            668        666     -2
- Misses           93         95     +2
  Partials          0          0

[][cc-pull]

Powered by Codecov. Last updated by [01de702...59dd8c0][cc-compare]
[cc-base-branch]: https://codecov.io/gh/TryGhost/Ghost-Desktop/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/TryGhost/Ghost-Desktop/compare/01de702bc1e28f0979e6fe6ed397a73ee5223648...59dd8c0a8f61a85d579055d6e367817dd4ca3299
[cc-pull]: https://codecov.io/gh/TryGhost/Ghost-Desktop/pull/153?src=pr


sql connection using service principal to specific table

I try to authenticate using service principal client id and secret.
I used the following:
    def credential_provider():
        config = Config(
            host=f"https://{server_hostname}",
            client_id=databricks_client_id,
            client_secret=databricks_client_secret)
        return oauth_service_principal(config)

    with sql.connect(server_hostname=server_hostname,
                     http_path=http_path,
                     credentials_provider=credential_provider) as connection:
        cur = connection.cursor()

and received the following logs with error" Error during request to server".
Do my authentication values the issue? or something else?
retry parameter: _retry_delay_min given_or_default 1.0
retry parameter: _retry_delay_max given_or_default 60.0
retry parameter: _retry_stop_after_attempts_count given_or_default 30
retry parameter: _retry_stop_after_attempts_duration given_or_default 900.0
retry parameter: _retry_delay_default given_or_default 5.0
Sending request: TOpenSessionReq(client_protocol=None, username=None, password=None, configuration={'spark.thriftserver.arrowBasedRowSet.timestampAsString': 'false'}, getInfos=None, client_protocol_i64=42247, connectionProperties=None, initialNamespace=None, canUseMultipleCatalogs=True, sessionId=None)
Error during request to server: {"method": "OpenSession", "session-id": null, "query-id": null, "http-code": null, "error-message": "", "original-exception": "invalid_client: Client authentication failed", "no-retry-reason": "non-retryable error", "bounded-retry-delay": null, "attempt": "1/30", "elapsed-seconds": "0.353135347366333/900.0"}
retry parameter: _retry_delay_min given_or_default 1.0
retry parameter: _retry_delay_max given_or_default 60.0
retry parameter: _retry_stop_after_attempts_count given_or_default 30
retry parameter: _retry_stop_after_attempts_duration given_or_default 900.0
retry parameter: _retry_delay_default given_or_default 5.0
Sending request: TOpenSessionReq(client_protocol=None, username=None, password=None, configuration={'spark.thriftserver.arrowBasedRowSet.timestampAsString': 'false'}, getInfos=None, client_protocol_i64=42247, connectionProperties=None, initialNamespace=None, canUseMultipleCatalogs=True, sessionId=None)
Error during request to server: {"method": "OpenSession", "session-id": null, "query-id": null, "http-code": null, "error-message": "", "original-exception": "invalid_client: Client authentication failed", "no-retry-reason": "non-retryable error", "bounded-retry-delay": null, "attempt": "1/30", "elapsed-seconds": "0.35685276985168457/900.0"}

Windows 10

Does the service principal itself work? If you take your config and do
w = WorkspaceClient(config)
w.current_user.me()

does it succeed?

I'm not sure if this is related or not, but it might help others that are facing a similar issue.
We started getting the same error (invalid_client: Client authentication failed) a few days ago, nothing has changed on our code base/infrastructure and it was working perfectly before.
For context, we're using Azure Databricks and connecting via service principal. We've been using the client id and secret from Microsoft Entra ID.
After digging a little bit on the stack trace, we confirmed the issue was coming from the call to get an access token, which should be something like this, according to the docs:
curl -X POST \
<per-workspace-url>/oidc/v1/token \
-d "grant_type=client_credentials" \
-d "scope=all-apis" \
-u "<service-principal-id>:<oauth-secret>"

We've then tried the same request using the same credentials from Microsoft Entra ID and confirmed the error was coming from this call.
Reading a bit more the docs we saw on Step 6: Create an Azure Databricks OAuth secret for the service principal that we should be using a Databricks OAuth Secret (apparently something new, since we've been using only the service principal credentials from Microsoft Entra ID). So we have it a try... I created a new token, sent it on the same request as before and it worked fine.
We believe this was introduced by this change, however we're not sure if this was an intended (and miscommunicated) breaking change or a regression bug, because it did work fine before 🤷🏼


SQL support for nested groupBys.

Allows, for example, doing exact count distinct by writing:
SELECT COUNT(*) FROM (SELECT DISTINCT col FROM druid.foo)

Contrast with approximate count distinct, which is:
SELECT COUNT(DISTINCT col) FROM druid.foo

👍

@jihoonson, on your test testRecursivelyNestedGroupby, that didn't work because it needs more merge buffers than the test allows, and since the merge buffer pool is blocking, the test blocks forever. I pushed a commit that adds your test, bumps up the test merge buffer pool to 3, adds a maxQueryCount config, and adds docs warning users about this potential deadlock in prod.
The doc blurb is:

Note that groupBys require a separate merge buffer on the broker for each layer beyond the first layer of the groupBy. With the v2 groupBy strategy, this can potentially lead to deadlocks for groupBys nested beyond two layers, since the merge buffers are limited in number and are acquired one-by-one and not as a complete set. At this time we recommend that you avoid deeply-nested groupBys with the v2 strategy. Doubly-nested groupBys (groupBy -> groupBy -> table) are safe and do not suffer from this issue. If you like, you can forbid deeper nesting by setting druid.sql.planner.maxQueryCount = 2.

@jihoonson, I just pushed commits for the rest of your comments, please let me know what you think. thanks for the review.

Raised #3819 for the deeply nested groupBy thing.

Thanks! The latest patch looks good to me.

Resolved conflicts.

:+1:


error with grounded_sam_simple_demo.py

Hello! Do you use or latest code or early code, would you like to check the detections output and offer us more info about this params, we will check this later

@rentainhe I am using code that was only downloaded from the git clone a few hours ago

hi there @Lufffya  just wondering whether you haved this resolved? I also have this issues...
Thank you!


DOM7 $$.ajax deletes POST fields that are null.

This is a (multiple allowed):

[x] bug

Framework7 Version: 1.4.2

Platform and Target: IOS

Live Link or JSFiddle/Codepen:

What you did
I am submitting to a server process that requires all fields in the POST. I include the field called {barcode: ''} . I expect that it will send all the fields in sparams with the additional barcode and SKU fields in the data:sparams object .  Code is as follows
var sparams = {barcode:"",SKU:'',_search:'true',nd:'1475552290823',rows:'15', page:'1', sidx:'SKU',sord:'asc'};
if(query.length == 18){
sparams.barcode = query;
sparams.SKU = "";
}
else{
sparams.barcode = "";
sparams.SKU = query;
}
console.log(sparams);
$$.ajax({
	url:"http://"+IP_ADDRESS+"/Pos/JSON_InventorySearch",
	method: "POST",
	data:sparams,
	processData: true,
	datatype: 'json',
	success: function (e,status,xhr) {...}
    });

}
Actual Behavior
When the POST is sent to the url, not all the fields are  included when xhr.send(postData) is called
I modified framework7.js so I can log the Send XHR
        // Send XHR
    console.log(postData);
        xhr.send(postData);

console reports this sent after I type in 154 in the search query
SKU=154&_search=true&nd=1475552290823&rows=15&page=1&sidx=SKU&sord=asc

Hi, can you please post a jsfiddle or codepen example so we can reproduce this issue? Thanks.
You can use one of these templates for that:

iOS JSFiddle/Codepen
Material JSFiddle/Codepen

Note: Please post your runnable example within 1 week, otherwise this issue will be closed.