Error when I try to make the dance motion of Decided

What assets can I download instead of trying to generate an SSR?
In case if you can't read the text on the image, here  is what it says:
System.IndexOutOfRangeException: Index was outside the bounds of the array.    at OpenMLTD.MillionDance.Core.VmdCreator.CreateBoneFrames(IBodyAnimationSource mainDance, PrettyAvatar avatar, PmxModel pmx, ScenarioObject baseScenario, ScenarioObject formationInfo, IBodyAnimationSource danceAppeal, Int32 formationNumber, AppealType appealType) in C:\projects\mltdtools\src\MillionDance\Core\VmdCreator.Dance.cs:line 35    at OpenMLTD.MillionDance.Core.VmdCreator.CreateDanceMotion(IBodyAnimationSource mainDance, ScenarioObject baseScenario, ScenarioObject formationInfo, PrettyAvatar avatar, PmxModel mltdPmxModel, IBodyAnimationSource danceAppeal, Int32 formationNumber, AppealType appealType) in C:\projects\mltdtools\src\MillionDance\Core\VmdCreator.Dance.cs:line 25    at OpenMLTD.MillionDance.MainWorker.DoWorkInternal(Object state) in C:\projects\mltdtools\src\MillionDance\MainWorker.cs:line 489    at OpenMLTD.MillionDance.MainWorker.DoWork(Object state) in C:\projects\mltdtools\src\MillionDance\MainWorker.cs:line 48

I think I see the issue, the Decided PST cards are not available on the game, because it says "Coming Soon", so once they are available, then it might work the next time I try.


squid configuration produced bungled localnet entry when an ipv6 ip alias is used

When using a network virtual IP alias with an ipv6 unique local unicast address fdxx::/64 the config backend produces an acl entry with a wrong prefix length:
acl localnet src fd42:1313:815:1::254/fd42:1313:815:1::254 # Possible internal network
this should be
acl localnet src fd42:1313:815:1::254/64
The malformed entry prevents squid to start.

@wytchmaster can you try https://github.com/opnsense/core/commit/20cf2eb64d70c06824316cfcb8c9ac2a73e59fcc ?
opnsense-patch 20cf2eb64d

I willbtry it tomorrow. Sorry for the delay, I was very busy on the job.

no problem, thanks in advance

looks good. The acl entry is well-formed now. Thanks.


Interaction matrix of user

Hey @siddz415 , just wanted to check in if you're able to respond to the comments in this PR? If you're too busy with other stuff that's totally cool. We were thinking we would re-assign the PR if there's not any motion before the meeting on Nov 7. Let us know if you're still interested in working on it.

Using latin-1 encoding in load_movie_titles() seems odd, but perhaps we're loading an ancient file which is not in utf8. It's all commented code in any case, so we should probably just delete those lines before merging to main.

The file is from 2007, so it's completely possible that it's not UTF-8. It's also possible that it doesn't have any non-ASCII characters anyway, so it might not matter.

The split() on , comma suggests that we might prefer to import csv.

These text files are not CSV, they are only CSV-like. Individual fields can have commas in them. Specifically movie_titles.txt often looks like:
1023, 1994, Forrest Gump
1024, 2001, Planes, Trains and Automobiles
...

In lightfm_recommendation.py and main.py we have a lot of code up at module level. Recommend burying it within def main(): or whatever. That way it will be safe for some future unit test to import it without side effects. The current code, which lacks a __main__ guard, is innocuous enough. But I'm concerned it will encourage folks to add stuff to this code base which is hard for a "second caller" (such as a unit test) to invoke. Such issues could be addressed in the current PR prior to a merge, or in a subsequent PR.

I agree. Better to put things in a main function now and figure out how to call it later. We can always rename main() to create_matrix() or whatever. I think a lot of this comes from the fact that we haven't really discussed how to organize files/modules/libraries/functions, so things are sort of in random places right now.

This PR is drifting toward being moribund. It is still mergeable at this point, though it proposes adding more than 4 million lines of CSV data, which for git is not quite a Best Practice.
It might be helpful to split Data from Code, merge the code, and move on to other issues and feature requests.

Thanks for updating the branch @jhanley634.
We discussed early on in our in-person meetings that our strategy should not be to commit the Netflix prize data. That's the reason we're exploring the other PR that automatically downloads it.
This has already been addressed in this comment: https://github.com/noisebridge/MediaBridge/pull/10#discussion_r1730052989

I guess my greater question is, what are in these mv files exactly?
 (Presumably those that rated that movie?)
Then the second one is user ids and dates?
Where is the rating (presumably on a 1-5 scale?) supposed to come from?

Okay we're clearly working on this at the exact same moment, which probably isn't a great idea. Please pull. I consolidated all of the files and put them in a sane place.

Okay we're clearly working on this at the exact same moment, which probably isn't a great idea. Please pull. I consolidated all of the files and put them in a sane place.

Will do! Hopefully didnt create some kind of merge conflict thing, but yeah will make sure to avoid any future interactions


CZML referenceFrame cleanup

add missing documentation for types that take referenceFrame as constructor parameter
Make logic for reference frame selection more clear.  defaultValue(foo, undefined) is misleading.
Add specific test that ensures default referenceFrame is FIXED (as specified in the CZML jsonschema)

This is all fine with me. @emackey please merge unless you still have some concerns.  Thanks.


Correct wrong readme file

Coverage remained the same at 93.082% when pulling 511e1376f6a6ed66af28627db48e04feca4b70c4 on razvanvasile:install_to_pypi into 7a2a29e3322d7e098838e7ed0d71cd292e026750 on willrogers:master.


Application screen shift up (on Android device and emulator)

Ionic version:  (check one with "x")
[ ] 1.x (For Ionic 1.x issues, please use https://github.com/ionic-team/ionic-v1)
[ ] 2.x
[x] 3.x
I'm submitting a ...  (check one with "x")
[x] bug report
[ ] feature request
[ ] support request => Please do not submit support requests here, use one of these channels: https://forum.ionicframework.com/ or http://ionicworldwide.herokuapp.com/
Current behavior :
Application screen shift up when device keybord is shown.
Then the offset remains until the application is launched again.
When the keyboard is not displayed, there is no offset.
Steps to reproduce :

Select / Tape / Focus on any input field.

Other information:

Tested on two Android real devices
tested on many android emulatotrs

Screen capture before focus on input (keyboard is shown)

Screen capture after focus on input (keyboard is shown)

Ionic info: (run ionic info from a terminal/cmd prompt and paste output below):
global packages:
    @ionic/cli-utils : 1.3.0
 You have been opted out of telemetry. To change this, run: cordova telemetry on. 7.0.1
    Ionic CLI        : 3.3.0

local packages:
    @ionic/app-scripts              : 1.3.7
    @ionic/cli-plugin-cordova       : 1.3.0
    @ionic/cli-plugin-ionic-angular : 1.3.0
    Cordova Platforms               : You have been opted out of telemetry. To change this, run: cordova telemetry on.  android 6.2.3 browser 4.1.0
    Ionic Framework                 : ionic-angular 3.3.0

System:
    Node       : v6.10.3
    OS         : Windows 10

Hello, thanks for opening an issue with us. Would you be able to share a repo we could use to reproduce this issue?

+1

Hello all! As it seems it has been a while since there was any activity on this issue i will be closing it for now. Feel free to comment if you are still running into this issue. Thanks for using Ionic!


Azure Monitor Exporter - Refactor to avoid sampleRate on Exception.

Refactored TelemetryItem
Updated tests
Added ActivityEvent.Timestamp to Exception Telemetry generated from Activity.Events.
Validated time change with an Example below.

using (var activity = source.StartActivity("SayHello"))
            {
                activity?.SetTag("foo", 1);
                activity?.SetTag("baz", new int[] { 1, 2, 3 });
                activity?.SetStatus(ActivityStatusCode.Ok);
                Task.Delay(2000).Wait();
                activity.RecordException(new Exception("test message"));
            }

{"name":"Exception","time":"2022-09-14T22:34:00.3311452Z","iKey":"00000000-0000-0000-0000-000000000000","tags":{"ai.operation.parentId":"9a59996e5c4d932f","ai.operation.id":"5793f4d2ce1a1db7309a30b3bbec0d00","ai.cloud.role":"[my-namespace]/my-service","ai.cloud.roleInstance":"my-instance","ai.internal.sdkVersion":"dotnet4.8.4515.0:otel1.3.1:ext1.0.0-alpha.20220914.1"},"data":{"baseType":"ExceptionData","baseData":{"exceptions":[{"typeName":"System.Exception","message":"test message","stack":"System.Exception: test message"}],"ver":2}}}

{"name":"RemoteDependency","time":"2022-09-14T22:33:58.3081482Z","sampleRate":100,"iKey":"00000000-0000-0000-0000-000000000000","tags":{"ai.operation.id":"5793f4d2ce1a1db7309a30b3bbec0d00","ai.cloud.role":"[my-namespace]/my-service","ai.cloud.roleInstance":"my-instance","ai.internal.sdkVersion":"dotnet4.8.4515.0:otel1.3.1:ext1.0.0-alpha.20220914.1"},"data":{"baseType":"RemoteDependencyData","baseData":{"id":"9a59996e5c4d932f","name":"SayHello","duration":"00:00:02.0428860","success":true,"properties":{"foo":"1","baz":"1,2,3"},"ver":2}}}

API change check
API changes are not detected in this pull request.


livereload runs without explicitly using --livereload

Hi there,
First, thanks for a great plugin, a life saver!
I'm facing couple of issues:

When launching the app using cordova run --device, i.e. without explicitly using --livereload, the plugin still runs. I'm on OS X, latest cordova and node version 4.2.

Hmm, thanks for filing this issue. I was able to repro it.

For some reason, nopt parses the --device option as --devicesync. (devicesync is synonymous to --livereload).

I just issued a fix for this issue, try it again and let me know if things work fine for you.

The "CTRL+C" issue not working on ios is due to the way "ios-sim" works, there's no way to get around it on a version of cordova-ios less than 3.9.2.

@omefire Yup that worked, thanks for the fix!

Cool. @1N50MN14 let me know if you find any other issues.


My version of moto is '4.0.11', and my version of boto3 is '1.26.23'
I installed moto using pip install moto[lambda]. I also tried pip install moto[awslambda] and pip install moto[all].

I saw this past issue (https://github.com/spulec/moto/issues/3567), but it didn't seem to have a clear resolution. Would love to get some help on this!

Pip is installing it in c:\python310\lib\site-packages, but VSCode uses a different Python installation/environment: ..\..\..\..\appdata\local\programs\python\python38\lib\site-packages. That's why it can't find Docker.
I would suggest running Pip in the Python installation that VSCode uses, or changing VSCode to point to c:\python310\lib.
BTW, it's unrelated to this, but you can get rid of the invalid distribution warnings:
https://stackoverflow.com/questions/67442661/warning-ignoring-invalid-distribution-ip-c-python39-lib-site-packages-how-d

Ah I see. That fixes it. Thanks a lot for your help!


Feature visualization

add feature visualization to netron
you select/upload a image from the file system. When you sleect a layer, netron displays the features that maximise the activation of the selected layer
based on the following research paper:
https://arxiv.org/pdf/1311.2901.pdf
there was a similar repo (https://github.com/keplr-io/quiver), but it was written for tensorflow 1 and it fails to construct valid maps of the models constructed with modern tools
this might be network-specific, so maybe implement slowly?

@ghylander can you implement this and submit a pull request.

i can try to implement support the feature visualization itself for caffe and maybe keras models, but i lack the knowledge to do the front end stuff (select file, show activations on the app in the layer information tab)

@ghylander how would this work for Caffe or Keras running in the browser and without a Python backend. Also can you make sure this works for all frameworks.

ah i see, yes, it wouldn't be possible to have it runing in the desktop, since deconvolution/gradient ascent operations are required
i'm developing the feature visualization using Francois Chollet book as reference
this implmentation requires use of python and keras as one is basically running the model in inference model
reagarding all the frameworks, it would require a lot of work
as i'm sure you know, each frameworks does things in their own way. Feature visualization in caffe models is not the same as in keras. They dont have the same functions, models are structured differently...
it would require a per-framework function

Look into your system brand name setting tools an play store app.


Fixing decimal places in the WHO GHE csv download

Hi @larsyencken - I was wondering if you could help me out with this if you have time. I'm pretty sure I am rounding all of the variables in this dataset to either 0 or 2 d.p. and am reinforcing this in the display sections of the meta.yml. However, when I download the csv of a variable (e.g. this one), it is giving many, many more decimal places!
I'm not sure if I'm doing something wrong?

@spoonerf taking a look, I don't think it's your fault. The reason it's doing that is likely because repacking lowers precision for floats and it somehow gets messed up on grapher upserts.

Hi @Marigold - think this is okay to merge now but thought I should pass it by you first!


Error during compilation - missing dependencies

Hi everybody,
i've cloned the repo and i'm trying to compile the source, but i have many errors of missing dependencies :
M2_REPO/com/comphenix/executors/BukkitExecutors/1.0.0/BukkitExecutors-1.0.0.jar
M2_REPO/com/comphenix/protocol/ProtocolLib/3.6.3-SNAPSHOT/ProtocolLib-3.6.3-SNAPSHOT.jar
M2_REPO/com/google/code/gson/gson/2.3/gson-2.3.jar
M2_REPO/net/ess3/Essentials/2.13.1/Essentials-2.13.1.jar
M2_REPO/net/milkbowl/vault/Vault/1.4.1/Vault-1.4.1.jar
M2_REPO/org/yaml/snakeyaml/1.14/snakeyaml-1.14.jar
I'm a beginner in java dev, does this lib must be present for building the JSON-API.JAR ?
if true, where can i get these lib ?
Thanks for your help
Best regards
cab

UP

Compiling is done via a maven project and if you are using an IDE such as eclipse you will need to install their maven integration plugin so that it can download the dependencies properly


openAnalyzer: false is ignored

Describe the issue
with the configuration
      analyzer({ analyzerMode: 'static', openAnalyzer: false }),

the server starts, instead of finishing the build
Expected behavior
It still runs the server, because preferOpenServer is true even if openAnalyzer is false
https://github.com/nonzzz/vite-bundle-analyzer/blob/7da1da7000d64b936d9e07ec18986a9b50d18699/src/server/index.ts#L71-L72
Steps to reproduce
Just run the build with this configuration.
Test case
No response
Additional comments
No response

I understand starting the server and opening browser are slightly different things, but i expect "static" mode to not start the server.
I believe the fix would be just to add the condition
const preferOpenServer = opts.analyzerMode === 'server' || (opts.analyzerMode === 'static' && opts.openAnalyzer)

@vxsx You're right. my mistake.

Yup, works with the new version. Thanks a lot!


[RFE] Use systemd-repart and dm-verity

Current situation
We build plain squashfs images
Impact
We don't benefit from dm-verity protection
Ideal future situation
Use systemd-repart to create dm-verity images

I have recently come to use mkosi for use-cases like this as it allows easy configuration and supports many neat features such as systemd-repart, automatic dm-verity (including root hash, signature and such), automatic generation of the extension-release file and such.

For what it's worth, cons: mkosi does not support Gentoo as a target, it's not neatly packaged and it breaks fairly often (being "hacked together" mostly with systemd's testing in mind).
Don't get me wrong: it's a neat tool that, in theory, would remove most friction when it comes to producing systexts/portableexts but those cons make it very hard to really rely on it in production.


Handles missing doc in npm change stream

This is a PR for issue #4506. Please let me know if this is what you had in mind.

@whit537 ready for review

Rebased, was b2570f1028e750e69b62fd6670ba0d368f34514e.

@kaguillera If you're good with 864a8396bcf32ac3d7dd56bf971d9ce230ab28e9 I'm ready to go.

Make sense. Once it is green we can merge.

!m @dmk246 First merge! 😄 💃  🌻


Edit a profile with .yaml

Hello to everyone.
I have a .yaml file.
When i am doing
sudo lxc profile edit routed_<IP_ADDRESS> < net_<IP_ADDRESS>.yaml
routed_<IP_ADDRESS> is configured well
config:   user.network-config: |     version: 2     ethernets:         eth0:             addresses:             - <IP_ADDRESS>/32             nameservers:                 addresses:                 - <IP_ADDRESS>                 search: []             routes:             -   to: <IP_ADDRESS>/0                 via: <IP_ADDRESS>                 on-link: true description: Default LXD profile devices:   eth0:     ipv4.address: <IP_ADDRESS>     nictype: routed     parent: wlp6s0     type: nic name: routed_<IP_ADDRESS> used_by: []
When i am trying the same inside a python script it doesn't work.
command4LXD = ['sudo','lxc','profile','edit','routed_'+possibleIP,' < ',name4yaml]
subprocess.run(command4LXD)
It prints :
...........................................................................
Description:
Edit profile configurations as YAML
Usage:
lxc profile edit [:] [flags]
...................................................................................................
When i am doing
command = "sudo lxc profile edit " + "routed_"+possibleIP + " < " + name4yaml
os.system(command)
it works , but when i am doing
sudo lxc profile show  routed_<IP_ADDRESS>
the output looks like this
config: {} description: "" devices: {} name: routed_<IP_ADDRESS> used_by: []
Any help ? Can i achieve the same result using pylxd ?
Thanks in advance !!

Yes, this kind of thing can be done with pylxd.  You want the profile part of the API which is detailed here: https://github.com/lxc/lxd/blob/master/doc/rest-api.md#10profiles
You use the profiles part of pylxd: https://pylxd.readthedocs.io/en/latest/profiles.html
Use the ipython repl to explore what you can do with the API; it'll give you a feel for how to achieve things.  Code that uses the profile: https://opendev.org/x/nova-lxd/src/branch/stable/rocky/nova/virt/lxd/flavor.py  -- it's a bit hairy but focus in the client.profile.* bits.
The subprocess.run() didn't work as you can't use a pipe (<) in the list passed to run() ... it just doesn't work that way.  Please read up on how subprocess works, and how you can pipe stuff to and from it.
Closing as it's not a bug.  Please re-open if you find a bug related to this behaviour.  Thanks.


Website needs a Header

Website needs a heder with name and logo of website

I can add a header to the web site and i can also add section of about us and contact us

@Dev-tanay I'd like to contribute to this project by adding the Header to the Website. This will results in enhancing the interactivity and improving the overall visuals of the website. Assigning this task to me would be fantastic.

I like to contribute on this project by designing header to website effectively. Please assign me for this project.

I would like to contribute on providing the header to website which contains the name and logo also ill be adding the background color as wheat so that it looks more effective or maybe a background effect please assign me this work


[Beta 2.0.0] SuperTable support in 2.0 beta

A question instead of an issue, but is expected that I am unable to map SuperTable rows in the beta?

If this is better to file in the SuperTable repo let me know, and I'll post there instead.
Thanks!
I am really liking the FeedMe updates.

Yep, gotta put together implementation for Super Table! Should be within the next week.
https://github.com/engram-design/SuperTable/issues/93

Implemented in the latest master. Will roll into the next release after some peer testing, so go ahead when you can!


Online and registered users count not up to date

Hello,
The slack online users count and total registered user count is not up to date with the actual online/registered count.
I have this issue on this page where it says 9/45 , but it actually currently has 18/252.
Can I do something to fix this ?
Thank you,

Hi, I am seeing this same issue? Did you find any solution to this?

Found a fix. Restarted the slcaking process.


Add prompt content API and example in Python

Added prompt content Python API
Added example in the notebook
Fixed some minor typos
Use Index instead of Search to get insights for known video
Added missing .env.example file
Added missing CONTRIBUTING.md file

@microsoft-github-policy-service agree company="Microsoft"


IdleClientTimeoutSeconds

In the summary of the IdleClientTimeoutSeconds property, it's stated that the timer resets when a message is received from a client OR a message is sent to the client. On the "Disconnecting Idle Clients" section of the readme, it's said that the timer is only reset when receiving a messages from the client.
How it's put in the readme seems to be the correct current behaviour. Is the summary just incorrect/out of date?

These values are just a pass through to the underlying TCP implementation. In your case with NETCore 3.1 it's passing these into SetSocketOption. In my experience, the behavior varies across platforms.
https://learn.microsoft.com/en-us/dotnet/api/system.net.sockets.socket.setsocketoption?view=net-8.0


Add image example to README and clean up a few tests

@ayush-1506 Be good if you look over the image example, thanks.


Infinite loop with syn::parse_str::syn::Expr("q^0:");

You can reproduce this bug with default features by calling
syn::parse_str::<syn::Expr>("q^0:");
This bug does not happen with feature full as this case seems handled here https://github.com/dtolnay/syn/blob/master/src/expr.rs#L1355
A fix could be to remove the check for input.peek(Token![as]) at https://github.com/dtolnay/syn/blob/master/src/expr.rs#L1402
(but I am not sure if it breaks other things)

Thanks! I published a fix in 1.0.66.


Does gotun2socks work on macOS?

There is a tun_linux.go file in bin directory, but missing tun_darwin.go.

I think the package itself works on macOS, as the code does not rely on OS-specific structs/interfaces. Thought I did't test.
The executable in bin directory only works on Linux. But it should be not hard to make it works on macOS. The only thing missing is a procedure that opens a tun device and returns a os.File object.

I will have a try, and give my feedback ASAP.

@liudanking gotun2socks works on macOS now. You can have a try. The problem only sets up the utun interface, you need to take care of routing yourself.

Could you give me a route command example for OSX? I am not familiar with OSX route command. I have try some commands below.
As you see.

Generaly the process includes changing default route to the tun/tap device, and exclude IP addresses of remote servers to go through the original network device so that traffic forwarded from local SOCKS5 proxy to remote servers would not loop back.

Follow command work for me. But I do not know it is corrent or not?
# start shadowsocks server on vps with udp relay
# start shadowsocks client on your computer with udp relay

sudo ./gotun2socks --local-socks-addr <IP_ADDRESS>:1080

sudo route add -net <IP_ADDRESS> <IP_ADDRESS> -netmask <IP_ADDRESS>
sudo route add -net <IP_ADDRESS> <IP_ADDRESS> -netmask <IP_ADDRESS>

orig_gw=$(netstat -nr | grep --color=never '^default' | grep -v 'utun' | sed 's/default *\([0-9\.]*\) .*/\1/' | head -1)
sudo route add -net my_ss_server $origin_gw -netmask <IP_ADDRESS>

@FlowerWrong I think the commands you posted should be sufficient. The 2th and 3th commands change default route to the new utun device. The 4th command extracts previous default gateway. The 5th command specifies your ss_server to go through the previous gateway.


In what order does 'flow_from_directory' function in keras takes the images?

I'm trying to classify images into 10 classes. To get probabilities for images, I'm using model.predict_generator() function in keras. This returns only prediction values and not the corresponding sample ID(In this case image file name).
test_datagen = ImageDataGenerator(rescale=1./255)

validation_generator = test_datagen.flow_from_directory(
        '/path/',
        target_size=(256, 256),
        batch_size=32,`
        class_mode='categorical')

predictions = model.predict_generator(validation_generator, val_samples=10000)

How do I find the corresponding image name/id of the predictions?
(OR)
In what order does the '.flow_from_directory' read the samples?
Does the 'batch_size' and 'val_samples' arguments affect the order of predictions?
Click here for code

Use validation_generator.class_indices

@Vivek-B Were you able to resolve the issue? I am also facing a similar problem.
@marcj Do those functions exist in Keras? I tried searching for those, but did not find any matching search results.

@raghuramdr Yes. marcj's suggestion was useful.
Type 'validation_generator.' and then

@Vivek-B Thanks for the answer. How do you find the order in which flow_from_directory take the images. I am asking this because the accuracy computed from predict_generator and that obtained from a confusion matrix do not match.

@raghuramdr I suppose that the order like in generator.filenames list because it is used for iteration in next() method.

i wish there was an example in the documentation for how to get test data into a csv file

@pengpaiSH @Vivek-B just to confirm:
Doing
predictions = model.predict_generator(validation_generator, val_samples=total_samples)
will go trough the whole dataset once and only once (assuming there are total_samples in the folder). To each prediction the 'real class' is validation_generator.classes.
I want to use this information to build a confusion matrix

I have exactly the same problem:

trying to classify images into N classes. The model is well trained and have ~97% accuracy

my generator.class_indices are:
{'classXX': 4, 'classXX': 6, 'classXX': 8, 'classXX': 3, 'classXX': 1, 'classXX': 7, 'classXX': 5, 'classXX': 2, 'classXX': 0}
classXX - are all different from each other
But when i predict a single image (even from training or validation set) my predictions are not corresponding to classes in class_indices
model.predict(x) returns some data, that do not correlate to class_indices above

Something that helped me, was to use a new generator each time. So if I would do
gen = image.ImageDataGenerator(shuffle=False, ...).flow_from_directory(...)
preds = model.predict_generator(gen, len(gen.filenames)

I would work as expected, but if then I use the same gen again for some other task it will mix everything up (even if shuffle=False).

@hdmetor ohhh thanks for your information!
This do help me a lot...
I always got different result when I was using evaluate_generator and predict_generator.
Now I create image data generator again before using predict_generator, they have the same result finally..(Not totally the same but make more sense)

Guys I had a question about how the function validation_generator makes the labels..... what is actually happening under the hood.

gen = image.ImageDataGenerator(shuffle=False, ...).flow_from_directory(...)
preds = model.predict_generator(gen, len(gen.filenames)

This worked for me. I set up a test data directory with class folders and the test images in them.  Although if I use model.predict on a single image I get totally different predictions.
Any ideas?

just another comment, using keras 2.0.6 in a kaggle competition I see the same iuuse
If I use the code below to generate predictions, I get correct predictions the first time I call it, and apparently shuffled predictions on the second call
`test_gen = validgen.flow_from_directory(
test_data_dir,
target_size=(img_height, img_width),
batch_size=1,
class_mode=None,
    class_mode='binary',
    seed=0,
    shuffle=False)

preds = model_final.predict_generator(test_gen, 1531)
print (preds[0:10])
print (test_gen.filenames[0:10])
preds = model_final.predict_generator(test_gen, 1531)
print (preds[0:10])
print (test_gen.filenames[0:10])`
Found 1531 images belonging to 1 classes. [[  1.00000000e+00]  [  1.81016767e-05]  [  9.99988794e-01]  [  8.15555453e-03]  [  3.15029087e-04]  [  6.08354285e-02]  [  5.00569877e-04]  [  6.83458569e-03]  [  9.88739491e-01]  [  2.21080336e-04]] ['x/1305.jpg', 'x/570.jpg', 'x/508.jpg', 'x/1076.jpg', 'x/624.jpg', 'x/94.jpg', 'x/1128.jpg', 'x/137.jpg', 'x/795.jpg', 'x/555.jpg'] [[  1.81016931e-05]  [  9.99988794e-01]  [  8.15555453e-03]  [  3.15029087e-04]  [  6.08354136e-02]  [  5.00570342e-04]  [  6.83457917e-03]  [  9.88739491e-01]  [  2.21080118e-04]  [  1.81115605e-02]] ['x/1305.jpg', 'x/570.jpg', 'x/508.jpg', 'x/1076.jpg', 'x/624.jpg', 'x/94.jpg', 'x/1128.jpg', 'x/137.jpg', 'x/795.jpg', 'x/555.jpg']

@brad0taylor It seems the generator.filename is always the same......but generator's output is not always orderd as that... It's so annoying.... so is there a explanation???????????????? @fchollet

Anyone found a usable solution to this? At least I know that I am not crazy :D

no problem @XRarach
I think there is still no official support for that. another thing I did in the past (which is NOT efficient) is to grab the file path from the generator and then pass each one through the network. Obviously will work only for reasonable small datasets.

The listed filenames are fixed.
https://github.com/fchollet/keras/blob/22e6bea8c2e23c6bbd6d98b4d3fe8b2e74c33c3d/keras/preprocessing/image.py#L1008
https://github.com/fchollet/keras/blob/22e6bea8c2e23c6bbd6d98b4d3fe8b2e74c33c3d/keras/preprocessing/image.py#L1004
https://github.com/fchollet/keras/blob/22e6bea8c2e23c6bbd6d98b4d3fe8b2e74c33c3d/keras/preprocessing/image.py#L859
The enumerated filenames are randomly shuffled.
https://github.com/fchollet/keras/blob/22e6bea8c2e23c6bbd6d98b4d3fe8b2e74c33c3d/keras/preprocessing/image.py#L1023
https://github.com/fchollet/keras/blob/22e6bea8c2e23c6bbd6d98b4d3fe8b2e74c33c3d/keras/preprocessing/image.py#L704
https://github.com/fchollet/keras/blob/22e6bea8c2e23c6bbd6d98b4d3fe8b2e74c33c3d/keras/preprocessing/image.py#L709

About this issue, what about overwritting the "filenames" parameter, right after building the batches ?
From this:
https://github.com/fchollet/keras/blob/4a28dc5debcd0bd790ef29d202342184fde5a1f4/keras/preprocessing/image.py#L1101-L1110
to something like this
       filenames = []
        for i, j in enumerate(index_array):
            fname = self.filenames[j]
            filenames.append(fname)
            img = load_img(os.path.join(self.directory, fname),
                           grayscale=grayscale,
                           target_size=self.target_size,
                           interpolation=self.interpolation)
            x = img_to_array(img, data_format=self.data_format)
            x = self.image_data_generator.random_transform(x)
            x = self.image_data_generator.standardize(x)
            batch_x[i] = x
        self.filenames = filenames

What do you think?

This thread has several topics, but I am having a similar problem as some of you.
TLDR: Simply call datagen.reset() before you call model.predict_generator() to get the same order as datagen.filenames and datagen.classes.
Explanation: I am using flow_from_directory() on my data-generator and I need to know which image is associated with each prediction. The problem is that the data-generator will output batches for eternity so even if it is not shuffled, it might start at different positions in the dataset each time we use the generator, thus giving us seemingly different outputs. So you would need to exactly match the number of iterations and batches to the dataset-size, in order for it to restart at the beginning of the dataset in each call to predict_generator(). You can see the internal batch-counter using datagen.batch_index.
Somebody suggested to create a new data-generator before each call to predict_generator(). That should work, but it is a hack-around and there is a better and simpler way, which is to call reset() on the generator, and if you have set shuffle=False then it should start over from the beginning of the dataset and give the exact same output each time, so that the ordering now matches datagen.filenames and datagen.classes. This solved the problem for me.
The last suggestion about updating the list of filenames would be a major side-effect and it should probably be avoided. If you need something like that and datagen.reset() is not enough for you, then add another variable to the datagen-object, but don't overwrite the existing list of filenames.

Just another comment, using keras 2.0.6 in a kaggle competition I see the same issue with the predict order.
If I use the code below to generate predictions, I get correct predictions the first time I call it, and apparently shuffled predictions on the second call:
> `test_gen = validgen.flow_from_directory(
>         test_data_dir,
>         target_size=(img_height, img_width),
>         batch_size=1,
>         class_mode='binary',
>         seed=0,
>         shuffle=False)
>
>
> preds = model_final.predict_generator(test_gen, 1531)
> print (preds[0:10])
> print (test_gen.filenames[0:10])
>
> pred1 = model_final.predict_generator(test_gen, 1531)
> print (pred1[0:10])
> print (test_gen.filenames[0:10])`

`Found 1531 images belonging to 1 classes.
[[  1.00000000e+00]
 [  1.81016767e-05]
 [  9.99988794e-01]
 [  8.15555453e-03]
 [  3.15029087e-04]
 [  6.08354285e-02]
 [  5.00569877e-04]
 [  6.83458569e-03]
 [  9.88739491e-01]
 [  2.21080336e-04]]
['x/1305.jpg', 'x/570.jpg', 'x/508.jpg', 'x/1076.jpg', 'x/624.jpg', 'x/94.jpg', 'x/1128.jpg', 'x/137.jpg', 'x/795.jpg', 'x/555.jpg']

[[  1.81016931e-05]
 [  9.99988794e-01]
 [  8.15555453e-03]
 [  3.15029087e-04]
 [  6.08354136e-02]
 [  5.00570342e-04]
 [  6.83457917e-03]
 [  9.88739491e-01]
 [  2.21080118e-04]
 [  1.81115605e-02]]
['x/1305.jpg', 'x/570.jpg', 'x/508.jpg', 'x/1076.jpg', 'x/624.jpg', 'x/94.jpg', 'x/1128.jpg', 'x/137.jpg', 'x/795.jpg',
```'x/555.jpg']`
why print "`Found 1531 images belonging to 1 classes."

Try validation_generator.class_indices and validation_generator.classes. pprint it and see how its useful to you.

Could you please give link of some example of a classification problem with more than two classes where these functions are used

@raghuramdr Yes. marcj's suggestion was useful.
Type 'validation_generator.' and then tab button. You'll get all the available functions.

Could you please give link of some example of a classification problem with more than two classes where these functions are used,with dataset used.I am trying to learn classification with more than two classes.I have a dataset with 100 classes in it  each containing 10 images,need to know how can i apply a cnn on it.

Try validation_generator.class_indices and validation_generator.classes. pprint it and see how its useful to you.

It worked for me, I wanted to see which class was assigned to which class like this:
print(validation_generator.class_indices)

Following could be done (for 2 classes) along with shuffle=False, batch_size = 32:

print("Number of Classes: ", training_set.num_classes)
print(training_set.class_indices)
cats_indx = np.where(training_set.labels == training_set.class_indices['cats'])[0][0]
dogs_indx = np.where(training_set.labels == training_set.class_indices['dogs'])[0][0]
print("First Index of Cat: ", cats_indx)
print("First Index of Dog: ", dogs_indx)
print("This is the 6th (indx = 5) batch: ", training_set[5][1])
finding the actual categorical label of our dogs
cat_batch_num = cats_indx//32
relative_index_of__first_cat = cat_batch_num % 32
cat_label = training_set[cat_batch_num][1][relative_index_of__first_cat]
finding the actual categorical label of our dogs
dog_batch_num = dogs_indx//32
relative_index_of__first_dog = dog_batch_num % 32
dog_label = training_set[dog_batch_num][1][relative_index_of__first_dog]
print("Cats = ", cat_label)
print("Dogs = ", dog_label)

Still having this issue with the latest version of tensorflow. Training on validation set gives 99% accuracy but when doing a predict it's all completely wrong, even if I take data from the training set itself. Any suggestions would be greatly apprecaited.

Still having this issue with the latest version of tensorflow. Training on validation set gives 99% accuracy but when doing a predict it's all completely wrong, even if I take data from the training set itself. Any suggestions would be greatly apprecaited.

Hi, I'm having these issue as well after using flow_from_directory method to predict images of two categories.  After calling model.predict_generator on test_generator, I print the predictions out and it seems all wrong at first glance. But after check  test_generator.filenames, I realize that it's correct because the loading sequence is not as expected. And it also matches the probability when I predict the images using another method.
you can try print out the sequence of how test_generator load the images by:
for file in test_generator.filenames:
print(file)
and then match the predictions with the files.
Good luck.


fix: Minting NFT - "dancing"

PR Type

[x] Bugfix
[ ] Feature
[ ] Refactoring

Context
if you open the signing modal the second time, that section is not fixed starts "dancing"

[x] Closes #11069

How to reproduce
<template>
  <SigningModal
    :title="'asd'"
    :is-loading="isLoading"
    :status="status"
  />

  <NeoButton @click="open">
    open
  </NeoButton>
</template>

<script lang="ts" setup>
import { NeoButton } from '@kodadot1/brick'
import type { TransactionStatus } from '@/composables/useTransactionStatus'

definePageMeta({
  layout: 'no-footer',
})

const isLoading = ref(false)
const status = ref(TransactionStatus.Unknown)
const opened = ref(false)

const sleep = (ms: number) => {
  return new Promise(resolve => setTimeout(resolve, ms))
}

const open = async () => {
  isLoading.value = true

  status.value = TransactionStatus.Casting
  await sleep(400)

  status.value = TransactionStatus.Broadcast
  await sleep(1000)

  status.value = TransactionStatus.Block
  await sleep(2000)

  if (opened.value) {
    return
  }

  status.value = TransactionStatus.Finalized
  opened.value = true
}
</script>

Screenshot 📸

[x] My fix has changed something on UI;

❌
https://github.com/user-attachments/assets/8261885e-27b6-4e8b-9436-e612f6f9f28b
✅
https://github.com/user-attachments/assets/adde6dca-6c1f-4654-ba08-0c577bf5d1da

cc @hassnian there is typo in the subtitle word

cc @hassnian there is typo in the subtitle word

fixed 0dfd77e


[HUDI-8522] Compaction using fg reader for partial update

Change Logs
Enable fg reader for compaction for spark and partial update is enabled.
Impact
Consistent usage of fg reader within hudi.
Risk level (write none, low medium or high below)
Medium.
Documentation Update
Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none".

The config description must be updated if new configs are added or the default value of the configs are changed
Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the instruction to make
changes to the website.

Contributor's checklist

[ ] Read through contributor's guide
[ ] Change Logs and Impact were stated clearly
[ ] Adequate tests were added if applicable
[ ] CI passed

Closing this in favor of #12390


add bower dependency, so that it does not have to be installed globally

Oopsy, turns out we actually still do need bower installed globally.
I stole this fix from uProxy:
https://github.com/uProxy/uproxy/blob/dev/setup.sh#L7

Tested on my Mac box as well as on my new Windows virtual machine, where I noticed this.

Got a verbal +1 from @mollyling .


SILGen: Base "currying" of functions on their lowered capture set, instead of their formal capture set.

This allows for slightly better codegen for nested functions that refer to other nested functions that don't transitively capture any local state, but more importantly, allows methods of local types to work while still referring to nested functions that don't capture local state, fixing rdar://problem/28015090.

@swift-ci Please smoke test

@slavapestov Does this look reasonable for 3.0?

Test failure looks like a test case that needs to be updated. I'll fix it.

@swift-ci Please smoke test

@swift-ci Please smoke test

Looks good, thanks for taking this one.


Iteration benchmarks

ONCE CELL               time:   [8.1764 ns 8.3092 ns 8.4391 ns]
SATIC INIT              time:   [4.6689 ns 4.6808 ns 4.6941 ns]
LAZY STATIC             time:   [3.6658 ns 3.6776 ns 3.6899 ns]

`ONCE CELL               time:   [8.1764 ns 8.3092 ns 8.4391 ns]
change: [-4.1588% -2.9220% -1.7032%] (p = 0.00 < 0.05)
Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
2 (2.00%) high mild
9 (9.00%) high severe
SATIC INIT              time:   [4.6689 ns 4.6808 ns 4.6941 ns]
change: [-0.4947% -0.1819% +0.1987%] (p = 0.30 > 0.05)
No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
LAZY STATIC             time:   [3.6658 ns 3.6776 ns 3.6899 ns]
change: [-0.1089% +0.2568% +0.6391%] (p = 0.19 > 0.05)
No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
`

ONCE CELL               time:   [7.4972 ns 7.5825 ns 7.6732 ns]
                        change: [-8.8147% -5.8075% -2.7453%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
  6 (6.00%) high mild
  8 (8.00%) high severe

SATIC INIT              time:   [3.8526 ns 3.8604 ns 3.8688 ns]
                        change: [-1.0426% -0.7681% -0.4908%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) low mild
  4 (4.00%) high mild

LAZY STATIC             time:   [3.4311 ns 3.4479 ns 3.4660 ns]
                        change: [-0.0633% +0.7792% +1.6084%] (p = 0.08 > 0.05)
                        No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild

LATE STATIC             time:   [8.0068 ns 8.1532 ns 8.3337 ns]
                        change: [+1.2641% +4.8297% +8.4336%] (p = 0.01 < 0.05)
                        Performance has regressed.
Found 17 outliers among 100 measurements (17.00%)
  3 (3.00%) high mild
  14 (14.00%) high severe

With inline never.
Change is inline never vs without
ONCE CELL               time:   [3.1872 ns 3.1942 ns 3.2020 ns]
                        change: [-57.759% -57.456% -57.169%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

SATIC INIT              time:   [3.4789 ns 3.4901 ns 3.5013 ns]
                        change: [-9.8885% -9.6186% -9.3120%] (p = 0.00 < 0.05)
                        Performance has improved.

LAZY STATIC             time:   [3.4367 ns 3.4703 ns 3.5145 ns]
                        change: [-1.5372% +0.2474% +2.9814%] (p = 0.84 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) low severe
  1 (1.00%) low mild
  2 (2.00%) high mild
  2 (2.00%) high severe

LATE STATIC             time:   [4.3129 ns 4.3212 ns 4.3311 ns]
                        change: [-50.384% -48.644% -46.990%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 15 outliers among 100 measurements (15.00%)
  3 (3.00%) low mild
  1 (1.00%) high mild
  11 (11.00%) high severe


ENH:  cumcount() for DataFrames and Series

Unless I am missing something obvious, we can do this:
df.groupby('x').cumcount()
but not:
df.cumcount()
Obviously, there are workarounds, but seems like we should be able to do on a dataframe/series if we can do on a groupby object?

an Index already does the
pls show a usecase

@johne13 you aren't confusing it with cumsum? Besides the index, you can also just do range(len(df))

It is not a super-common event, to be sure, but there are cases like this: that I bump into somewhat regularly:
http://stackoverflow.com/questions/36044890/how-to-speed-up-a-very-slow-pandas-apply-function/36047836#36047836
Simpler example:
df = pd.DataFrame({ 'x':[1,2,1,2] })
#df[ df.x == 2 ].cumcount()    # doesn't work of course

df.loc[ df.x == 2, 'y'] = 1    # so I do a two step with cumsum as 2nd step
df.y.cumsum()

Out[428]:
0    NaN
1    1.0
2    NaN
3    2.0

So yeah, lots of easy ways to do this on ENTIRE dataset but as soon as you want to do a selection, it falls apart.  Or again, maybe I am missing an easier way.
Also, I would think whatever justification there is for having this as a groupby method applies just as much to having it work for a dataframe or series, no?

the common idiom is
.reset_index()

no for a groupby this is expensive if u don't have an array impl (which is why it exists)

I guess I am missing something.  I couldn't do this, for example,
df['new'] = df[ df.x == 2 ].reset_index()
because obviously I have reset the index so can't correctly align it with the original dataset.
I should have been more clear about that aspect.  I am generally trying to create a new column when I mistakenly attempt to use the nonexistent cumcount method.

ok not averse to adding this method for consistency (among other cum. methods) but should return a Series
pretty trivial impl if u would like to submit a PR

signature would be identical to .cum* (at end of core/generic.py)
In [1]: pd.DataFrame.cumsum?
Signature: pd.DataFrame.cumsum(self, axis=None, dtype=None, out=None, skipna=True, **kwargs)
Docstring:
Return cumulative sum over requested axis.

Parameters
----------
axis : {index (0), columns (1)}
skipna : boolean, default True
    Exclude NA/null values. If an entire row/column is NA, the result
    will be NA

Returns
-------
sum : Series
File:      ~/pandas/pandas/core/generic.py
Type:      instancemethod

Potentially not as fast (?), and handles NAs differently from the case above, but very idiomatic:
In [12]: df.where(df==2).expanding().count()
Out[12]:
     x
0  0.0
1  1.0
2  1.0
3  2.0

Taken a step further: we could offer a argument for changing the NaN behavior and offer all the cum-x behavior with .expanding()

@MaximilianR actually that's a good point. would you make a PR to:

cross-link doc-strings for .cum* to the .expanding().* (for sum,prod,max,min, which are only supported).
if you find a nice place in computation.rst where you might want to mention these similarities.

maybe give a tiny example
In [7]: s = Series([1,2,np.nan,3,np.nan,4])

In [8]: s.cumsum()
Out[8]:
0     1.0
1     3.0
2     NaN
3     6.0
4     NaN
5    10.0
dtype: float64

In [9]: s.expanding().sum()
Out[9]:
0     1.0
1     3.0
2     3.0
3     6.0
4     6.0
5    10.0
dtype: float64

In [10]: s.cumsum().ffill()
Out[10]:
0     1.0
1     3.0
2     3.0
3     6.0
4     6.0
5    10.0
dtype: float64

@MaximilianR actually that's a good point. would you make a PR to:

I am jammed at the moment (although free enough to cruise the feed, it seems), but I can add that if no one else picks it up over the next few weeks

ok, let me make an issue so we don't forget.

@MaximilianR   I think you can get the same behavior as in my earlier example by using a boolean mask instead of where:
In [504]: df['new'] = df.x[ df.x == 2 ].expanding().count()

In [505]: df
Out[505]:
   x  new
0  1  NaN
1  2  1.0
2  1  NaN
3  2  2.0

Here's a quick speed test, cumsum about 4x faster than expanding().count()
In [515]: df = pd.DataFrame({ 'x':np.random.randn(10000) })

In [516]: %timeit df['new1'] = df.x[ df.x > 0 ].expanding().count()
100 loops, best of 3: 4.57 ms per loop

In [517]: %%timeit
     ...: df.loc[ df.x > 0, 'new2' ] = 1
     ...: df['new2'] = df['new2'].cumsum()
1000 loops, best of 3: 1.12 ms per loop

try assigning to a range ITS much faster still. this is the impl
we were not suggesting using expanding.count here it's doing a lot more work and very careful about min periods and such

@jreback  So do you still think cumcount is worthwhile or is expanding().count() the way to go?
If you still like the cumcount idea I'll give it a shot though I have not done a PR before.  Of course if someone else wants to do it I'd gladly defer to them.

I think it rounds out the cum functions a bit and would be ok

OK, I think I have an initial attempt and will post the code and example in a moment, but first want to list other similar functions as a point of comparison:

cumsum() -- outputs series or dataframe

(groupby)cumcount() -- outputs a series, numbered 0,1,2, ... n-1

expanding().count() -- outputs series or dataframe, numbered 1,2,3, ... n

Starting from there, it made the most sense to me to aim for something roughly consistent with cumsum() and expanding().count and not (groupby)cumcount().  Example to follow, see what you think.

Here's the code, added to generic, just after cummax (and also cumsum for comparison).
        cls.cumcount = _make_cum_function(
            'count', name, name2, axis_descr, "cumulative count",
            lambda y, axis: np.cumsum(~np.isnan(y), axis).astype(float), np.nan, np.nan)

        cls.cumsum = _make_cum_function(
            'sum', name, name2, axis_descr, "cumulative sum",
            lambda y, axis: y.cumsum(axis), 0., np.nan)

It's a little less natural than the other cum functions, but that was the best way I could come up with that fit the existing _make_cum_function() infrastructure.  It's basically cumsum but needs extra code to essentially convert non-nan to ones while maintaining the same treatment of nans as cumsum.
In some quickie timings it is just a little slower than cumsum() and much faster than expanding().count().  Exactly as expected, I think.
Examples:
df = pd.DataFrame({ 'x':[1,1,np.nan,1], 'y':range(4)  })

df.cumsum()
Out[18]:
     x    y
0  1.0  0.0
1  2.0  1.0
2  NaN  3.0
3  3.0  6.0

df.cumcount()
Out[19]:
     x    y
0  1.0  1.0
1  2.0  2.0
2  NaN  3.0
3  3.0  4.0

df.expanding().count()
Out[20]:
     x    y
0  1.0  1.0
1  2.0  2.0
2  2.0  3.0
3  3.0  4.0

The problem with making it consistent with groupby().cumcount() is that that function does not take the values into accound (whether they are NaN or not), it just numbers the entries in the group regardless of its values. That is also the reason that it returns a Series.
So if we want to make Series/DataFrame.cumcount consistent with the existing method, then it should also not disregard NaN values (as you do in your example implementation).
But, I suppose this defeats part of the reason you want a Series/DataFrame.cumcount?

@jorisvandenbossche I really hadn't thought about how exactly a Series/DataFrame.cumcount would work until I started comparing it to all the existing functions (count/cumsum/groupby.cumcount).  Then I realized it couldn't be consistent with all of them, only some.
Out of all the comparison functions, it seems to me that it ought to work approximately the same as expanding().count(), right?  And conversely, the way groupby.cumcount works makes the least sense to me, but of course I'll gladly defer to you and others on how to best make this consistent with the rest of pandas.

This is almost a trivial function, should be something like:
def cumcount(self, axis=0, skipna=True):
   ax = self._get_axis(axis)
   if skipna and ax.hasnans:
         result = ax.isnull().cumsum()
   else:
         result = range(len(ax))
    return Series(result, index=ax)

this is really just a range; if the index has nans then its slightly different.

I guess it technically should have an ascending arg to be consistent. Further .groupby.cumcount() should have skipna.
This should mirror the .cum* signatures.

@jreback But I would also find it very unfortunate that groupby().cumcount() and groupby().apply(lambda x: x.cumcount()) would not be the same ..
And it is, as far as I can see, not possible to have both that and have it consistent with Series.cumsum and others

Actually, to phrase the question in another way: should cumcount work on the index or on the values?
The example implementation of @johne13 works on the values (ignoring NaNs there), while the example implementation of @jreback works on the index.

@jorisvandenbossche actually that's a good point. it should work on the values, similarl to cum*.

Sorry, some of these details are over my head -- have never really delved into pandas internals before.  I tried to make it work as similarly to cum* as possible, that's why it is done via _make_cum_function like the other cum functions rather than a standalone.  Signature is comparable to cumsum.
In [28]: pd.DataFrame.cumcount?
Signature: pd.DataFrame.cumcount(self, axis=None, dtype=None, out=None, skipna=True, **kwargs)
Docstring:
Return cumulative count over requested axis.

Parameters
----------
axis : {index (0), columns (1)}
skipna : boolean, default True
    Exclude NA/null values. If an entire row/column is NA, the result
    will be NA

Returns
-------
count : Series
File:      ~/pandas-johne13/pandas/core/generic.py
Type:      function

And also regarding groupby.cumcount(), it actually doesn't agree with groupby.count() in that it's top value is n-1 whereas count returns n.

@johne13 no .count() is the number of non-nans, whereas .cumcount() is an enumeration.

@jorisvandenbossche actually that's a good point. it should work on the values, similarl to cum*.

And that's the problem, as for groupby().cumcount, it works on the index (actually, it is not really defined there, but since it gives back a series regardless of how many columns you have, it is like it would work on the index).
So it that sense, groupby().cumcount is not really a cumulative count, but rather a cumulative size .. (like you have both groupby().count and groupby().size)

Right, I'm just noting that the following holds true for the relationship between sum and cumsum
df.cumsum()[-1:] == df.sum()
The same relationship holds for prod/cumprod, max/cummax, and min/cummin.  So out of all these relationships, groupby().cumcount seems like the one that is somewhat inconsistent with other pandas functions.

Yeah, as Joris notes, groupby.cumcount seems more analogous to groupby.size than groupby.count except for the n vs n-1 aspect.

@jreback @jorisvandenbossche  FYI, I will abandon this for now, pending further instructions.  I'm not quite sure what if anything you guys decided about how best to proceed.  And this might not be a good first project for someone who has never done a PR and is not super familiar with all the underlying code, so if someone else want to do something with it I'm happy to step aside.

One other point of comparison for a hypothetical cumcount() is rank(method='first') on an array of ones.  Just playing around with this a little bit it seems to do approximately the same thing as expanding().count() but is much faster -- actually faster than a cumsum() in my test example but I imagine that doesn't hold in all cases since adding must generally be faster than sorting (?).

@johne13 almost everything is faster than sorting :) sorting O(n log n); people go to great length to avoid sort.

Right, except in this case it's "sorting" a column of ones, so it's just a matter of how long it takes the algorithm to confirm that it's already sorted, there is no actual sorting.
But I mainly mentioned rank() just as another function to consider in terms of consistency with how other existing functions behave.

So we have very fast routines to determine whether something is monotonic which determine if its 'sorted' very quickly.
In [2]: Index([1,2,3]).is_monotonic
Out[2]: True

In [3]: Index([3,2,1]).is_monotonic
Out[3]: False


Please consider fixing issues detected by repository checker

Notification from ioBroker Check and Service Bot
Dear adapter developer,
I'm the ioBroker Check and Service Bot. I'm an automated tool processing routine tasks for the ioBroker infrastructure. I have recently checked the repository for your adapter schedule-switcher for common errors and appropiate suggestions to keep this adapter up to date.
This check is based the current head revisions (master / main  branch) of the adapter repository
Please see the result of the check below.
ioBroker.schedule-switcher
   -
ERRORS:

[X] :heavy_exclamation_mark: [E251] Bluefox was not found in the collaborators on NPM!. Please execute in adapter directory: npm owner add bluefox iobroker.schedule-switcher

WARNINGS:

[X] :eyes: [W040] "keywords" within package.json should contain "ioBroker"
[ ] :eyes: [W401] Cannot find "schedule-switcher" in latest repository

SUGGESTIONS:

[X] :pushpin: [S522] Please consider migrating to admin 5 UI (jsonConfig).

Please review issues reported and consider fixing them as soon as appropiate.
Errors reported by repository checker should be fixed as soon as possible. Some of them require a new release to be considered as fixed. Please note that errors reported by checker might be considered as blocking point for future updates at stable repository.
Warnings reported by repository checker should be reviewed. While some warnings can be ignored due to good reasons or a dedicated decision of the developer, most warnings should be fixed as soon as appropiate.
Suggestions reported by repository checker should be reviewed. Suggestions can be ignored due to a decision of the developer but they are reported as a hint to use a configuration which might get required in future or at least is used be most adapters. Suggestions are always optional to follow.
You may start a new check or force the creation of a new issue at any time by adding the following comment to this issue:
@iobroker-bot recheck
or
@iobroker-bot recreate
Please note that I (and the server at GitHub) have always plenty of work to do. So it may last up to 30 minutes until you see a reaction. I will drop a comment here as soon as I start processing.
Feel free to contact me (@iobroker-bot) if you have any questions or feel that an issue is incorrectly flagged.
And THANKS A LOT for maintaining this adapter from me and all users.
Let's work together for the best user experience.
your
ioBroker Check and Service Bot
@mcm1957 for evidence
Last update at Mon, 28 Oct 2024 12:02:52 GMT based on commit 327d030783fc95605500d193d6dac27a7242f37b
ioBroker.repochecker 3.1.4

@iobroker-bot recheck

@ioBroker-Bot recheck

@iobroker-bot recheck

@iobroker-bot recreate


Any way to "thread" transaction history?

I love this package Jack. 3 quick questions:

I'm doing this for my update.
Cards.update(
{_id: id},
{$set: card},
{tx: true,
description:"funky edit",
context: {deckId:"abc", cardId:"123"}}
);
The context gets saved, but the "description" does not. Any way to change the description?

In my model, I'm working with metaphorical "documents". The multi undo/redo is great. But when the user switches from "doc" A to "doc" B... clicking undo in doc B starts reverting document A. Any way to filter transactions by "context" with the API or should I include your sources in my project and filter what the server publishes?

If I do #2, I can fix #1.  Will the undo/redo still "work"? All the tx records for a "mongo document" would still be processed sequentially. It would be really cool if .undo/redo() could take a parameter for "threadId" or "contextId". :)

Thank you!

@ericoe. So sorry I didn't reply here earlier:

The description you refer to would be the text that goes on the undo/redo buttons, right? context is a special-cased field, which is why it gets saved, but other arbitrary fields are not saved anywhere. To alter the description, use tx.start("funky edit"); Cards.update( ... ); tx.commit();.
This is do-able using the context, but not easily -- you'd need to write your own undo ui/logic. You can target specific transactions for undo/redo by writing tx.undo(transaction_id) or `tx.redo(transaction_id). To get the transaction_id, you'd have to filter the available transactions by something you've stored in the context of the transaction documents.
This is possible (I can think of a way), but it's not implemented yet.  Will leave this issue open to remind myself to do it.

Yes. Exactly. And I was thinking start/commit was overkill w/o looking at what was going on under the covers already.

2 & 3. Got it. Thanks. I'll play around with undo via tx ID and a custom filter and see what happens. :)

For 1., I agree that it's overkill.  I'm thinking of introducing a description field for single-action transactions that auto-commit.  Makes it quicker than wrapping the single mutator command in a start-commit block.  But it could also lead to confusion, as it would be ignored for a multi-action transaction.


Add bigmap dynamic navigation

Add bigmap dynamic navigation which would open the bigmap and reactively navigate as opposed to the current pre-determined movement

Closed as part of test 27 and test 28, see #110 for details


Separate common utils

The simplification implementation is almost complete. It’s working with the clean implementations of the face lattice and dependence raising.
This PR pushes some its changes to ISLUtil and other common functions. This also changes the FaceLattice to add a create methods that accepts ISLSets.

Looks good! Merging.


ansible task "setup-postgres.yml" requires "pkg-config"

Bug report
Describe the bug
I'm trying to run the ansible playbook against my own server.
The task "Postgres - configure" inside the "setup-postgres.yml" file was failing with a requirement of "pkg-config".
This was installing against a basic Ubuntu install from https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img
To Reproduce
create a basic VM using https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img with 16 cores and suitable memory.
Run the playbook against the VM.
Expected behavior
Not to error
To Fix
Adding the - pkg-config to the "Postgres dependencies" task of "setup-postgres.yml" solves this issue.
System information
Running ansible from a 4 core VM running the same Ubuntu image.
Additional context
I'm very new to ansible, so I don't know if there is a better place to include this dependency? Perhaps it's used in other tasks? I feel like it is used when compiling a lot of the postgres extensions.

Hi @Towerful - thanks for reporting this!
Also added pkg-config to setup-postgres.yml - https://github.com/supabase/postgres/pull/503


Vision API detectText returns TypeError: formatMethodMap[type] is not a function

Environment details

OS: MacOS
Node.js version: 7.6.0
yarn version: 0.21.3
@google-cloud/vision version: 0.8.0

Steps to reproduce
I have a testing script with sample images that worked before (i.e. successfully get the ocr text response). However, today I am having the same issue as reported in #1333 again.
Code:
let vision = require('@google-cloud/vision')({
  projectId: 'my-expense-keeper',
  keyFilename: './keyfile/keyfile.json'
});

import { Observable } from '@reactivex/rxjs';

const fs = require('fs-extra');

// const types = ['labels'];
// const types = ['faces'];
// const types = ['logos'];
const types = ['text'];

const obj = fs.readJsonSync('./test-files/receipt-image-req-1.json', {throws: false});

// console.log(JSON.stringify(obj));

let buffer = new Buffer(obj.content, 'base64');

let visionObs = Observable.fromPromise(vision.detect(buffer, types));

visionObs.subscribe(
  data => {
    console.log(`Cloud vision api response: ${JSON.stringify(data)}`);
    process.exit(1);
  },
  err => {
    console.log(`Cloud vision api error: ${JSON.stringify(err)}`);
  }
);

Where the content is base64 encoded image content.
When run the above code with type set to 'labels', I can get the following response:
[
  [
    "text",
    "label",
    "document"
  ],
  {
    "responses":[
      {
        "labelAnnotations":[
          {
            "mid":"/m/07s6nbt",
            "description":"text",
            "score":0.9377162
          },
          {
            "mid":"/m/05c0n6k",
            "description":"label",
            "score":0.591876
          },
          {
            "mid":"/m/015bv3",
            "description":"document",
            "score":0.5269188
          }
        ]
      }
    ]
  }
]

When run the above code with type set to 'labels', I can get the following error:
/Users/clarence/workspace/mek/mek-image-api/node_modules/@google-cloud/vision/src/index.js:538
          return formatMethodMap[type](annotation, options);
                                      ^
TypeError: formatMethodMap[type] is not a function
    at /Users/clarence/workspace/mek/mek-image-api/node_modules/@google-cloud/vision/src/index.js:538:39
    at Array.map (native)
    at decorateAnnotations (/Users/clarence/workspace/mek/mek-image-api/node_modules/@google-cloud/vision/src/index.js:548:16)
    at Array.map (native)
    at /Users/clarence/workspace/mek/mek-image-api/node_modules/@google-cloud/vision/src/index.js:409:10
    at /Users/clarence/workspace/mek/mek-image-api/node_modules/@google-cloud/vision/src/index.js:133:5
    at Object.handleResp (/Users/clarence/workspace/mek/mek-image-api/node_modules/@google-cloud/common/src/util.js:135:3)
    at /Users/clarence/workspace/mek/mek-image-api/node_modules/@google-cloud/common/src/util.js:466:12
    at Request.onResponse [as _callback] (/Users/clarence/workspace/mek/mek-image-api/node_modules/retry-request/index.js:120:7)
    at Request.self.callback (/Users/clarence/workspace/mek/mek-image-api/node_modules/request/request.js:186:22)

I have attached the image that I am using for your reference.
Thanks!
Clarence

Have the exact same issue since a few hours ago!

Same here

We are having the same issue. Happening for all images that contain text. We were forced to disable the service in production.

@jmuk

looks like this PR does a trick https://github.com/GoogleCloudPlatform/google-cloud-node/issues/2035, any way to fix this issue except patching the lib? or just wait for the lib to be updated?

+1

Vision has updated its API. I'm about to merge the code to handle the changes and we'll do some insta-releasing. Sorry for the trouble, everyone.

I'm having the same issue as well :) glad i found this thread.

All for one and one for all! 👍

Also seeing this issue while using https://www.npmjs.com/package/gcloud
Line numbers are different but error char location is the same.
BTW @stephenplusplus do you plan to update gcloud 0.47?

gcloud is deprecated, but we will update the bundled package, google-cloud: https://www.npmjs.com/package/google-cloud

Hey everyone, I've published a fix for this in<EMAIL_ADDRESS>please feel free to upgrade and let us know if the issue persists. Thanks!

I've test it and that worked :) @callmehiphop thank you :)

I've tested it too,.. It is working! Thanks @callmehiphop :)

I got the same problem today. @callmehiphop How can I upgrade the google-cloude/vision? Thank you!

npm install<EMAIL_ADDRESS>

@callmehiphop it looks like this may have resurfaced.  Your update worked beautifully for me until this morning.
I have double- and triple-checked that I'm actually running 0.8.1


service accounts rights are missing for tobs-timescaledb role

What happened?
During update to 17.17.0 from 17.15.0 there was a RBAC related error with the service account rights for the tobs-timescaledb role.
Did you expect to see something different?
The rights for the service account should be checked/confirmed. The error causes timescaledb to crashloop.
How to reproduce it (as minimally and precisely as possible):
Install 17.50.0 via helm, proceed with upgrade to 17.17.0
Environment
Kubernetes 1.21.14

tobs version:

17.15.0 with an upgrade to 17.17.0

Kubernetes version information:
kubectl version
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.1", GitCommit:"3ddd0f45aa91e2f30c70734b175631bec5b5825a", GitTreeState:"clean", BuildDate:"2022-05-24T12:26:19Z", GoVersion:"go1.18.2", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.14", GitCommit:"0f77da5bd4809927e15d1658fb4aa8f13ad890a5", GitTreeState:"clean", BuildDate:"2022-06-15T14:11:36Z", GoVersion:"go1.16.15", Compiler:"gc", Platform:"linux/amd64"}

tobs Logs:
2022-11-15 04:32:07,937 ERROR: create_config_service failed                                                                                Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/patroni/dcs/kubernetes.py", line 950, in _create_config_service
if not self._api.create_namespaced_service(self._namespace, body):
File "/usr/lib/python3/dist-packages/patroni/dcs/kubernetes.py", line 483, in wrapper
return getattr(self._core_v1_api, func)(*args, **kwargs)                                                                                File "/usr/lib/python3/dist-packages/patroni/dcs/kubernetes.py", line 419, in wrapper
return self._api_client.call_api(method, path, headers, body, **kwargs)
File "/usr/lib/python3/dist-packages/patroni/dcs/kubernetes.py", line 388, in call_api
return self._handle_server_response(response, _preload_content)                                                                                File "/usr/lib/python3/dist-packages/patroni/dcs/kubernetes.py", line 218, in _handle_server_response
raise k8s_client.rest.ApiException(http_resp=response)                                                                                patroni.dcs.kubernetes.K8sClient.rest.ApiException: (403)
Reason: Forbidden                                                                                HTTP response headers: HTTPHeaderDict({'Audit-Id': 'f4d1cc89-9193-4363-8ee4-be89d8ea2e2a', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Content-Type-Options':
'nosniff', 'X-Kubernetes-Pf-Flowschema-Uid': '34201944-ab28-47ee-8f64-6d9e778d06b0', 'X-Kubernetes-Pf-Prioritylevel-Uid': '1e12bbf7-cd0e-40cf-be91-2bd42febd6e0', 'Date': 'Tue, 15 Nov 2022 04:
32:07 GMT', 'Content-Length': '299'})                                                                                HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"services is forbidden: User \"system:serviceaccount:tobs:tobs-timescaledb\" cannot creat
e resource \"services\" in API group \"\" in the namespace \"tobs\"","reason":"Forbidden","details":{"kind":"services"},"code":403}\n'

Notes
I was able to proceed with an update when I manually modified and added the ability for the service account to alter services.

This error also happens during a new install of 17.17.0
2022-11-15 10:17:31,167 ERROR: create_config_service failed
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/patroni/dcs/kubernetes.py", line 950, in _create_config_service
    if not self._api.create_namespaced_service(self._namespace, body):
  File "/usr/lib/python3/dist-packages/patroni/dcs/kubernetes.py", line 483, in wrapper
    return getattr(self._core_v1_api, func)(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/patroni/dcs/kubernetes.py", line 419, in wrapper
    return self._api_client.call_api(method, path, headers, body, **kwargs)
  File "/usr/lib/python3/dist-packages/patroni/dcs/kubernetes.py", line 388, in call_api
    return self._handle_server_response(response, _preload_content)
  File "/usr/lib/python3/dist-packages/patroni/dcs/kubernetes.py", line 218, in _handle_server_response
    raise k8s_client.rest.ApiException(http_resp=response)
patroni.dcs.kubernetes.K8sClient.rest.ApiException: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Audit-Id': '6d0823fa-d627-4ab7-9342-894587a0b624', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'X-Kubernetes-Pf-Flowschema-Uid': '43fde79c-ff65-437e-b03c-edbdbb29013b', 'X-Kubernetes-Pf-Prioritylevel-Uid': '73f0f59e-1d05-46ee-8872-1f406aa1b018', 'Date': 'Tue, 15 Nov 2022 10:17:31 GMT', 'Content-Length': '299'})
HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"services is forbidden: User \\"system:serviceaccount:tobs:tobs-timescaledb\\" cannot create resource \\"services\\" in API group \\"\\" in the namespace \\"tobs\\"","reason":"Forbidden","details":{"kind":"services"},"code":403}\n'

This issue is explained in some detail here with a workaround.  I just re-deployed the latest Tobs in hopes that this error was fixed, but can confirm I'm still seeing the issue.
https://github.com/timescale/helm-charts/issues/405
After applying the recommended fix, the error does go away, however  I'm still not getting any data in Grafana...

This is/was a bug in Patroni which was fixed with https://github.com/zalando/patroni/issues/1132 & https://github.com/zalando/patroni/pull/2390.
We have merged the change to add these fixes into the timescale-db container image https://github.com/timescale/timescaledb-docker-ha/pull/319
The image changes are currently in the process of being tested.  I hope to have a release soon.

facing the same issue on latest version.

timescale/timescaledb-ha:pg14.6-ts2.8.1-p0 has been released with the Patroni fix.

I performed a fresh tobs installation with a modified my_values.yaml file that specified the new timescaledb image (timescaledb-ha:pg14.6-ts2.8.1-p0 and I still see the 403 error in the tobs-timescaledb-0 pod logs. The only item I changed in the tobs values file was tag: pg14.6-ts2.8.1-p0. I also confirmed this is the image that was pulled for tobs-timescaledb-0.

Yup you are correct, sorry about that.  Can you try again with this image and let me know.  timescale/timescaledb:pg14.6-ts2.8.1-patroni-static-primary-p0

I am currently no longer seeing the error logs when using that image.

timescale/timescaledb:pg14.6-ts2.8.1-patroni-static-primary-p0
I also no longer see the error in the logs after re-deploying with this image.  Thank you!

Awesome.  I am going to close this, please re-open if needed.


Type definiton for v6

Version
6.0.0-alpha.1
Description
Cannot find type definitions for v6 alpha version. @types/react-router & @types/react-router-dom are still for v5.

They haven't yet been written.


Optionally warn on unassigned names

From #97: I'm still unsure if we should do this, but let's separate the issues.
We could issue warnings on unassigned names, possibly from missing import statements. What I'm worried about is that lots of examples use unassigned dummy variables, which would raise false alarms. But if this is behind a flag like verbose or something else, it could be easier to deal with.
We could also try to find the location of the unassigned name, but this I think is too much guesswork.

As per #97, I think the configuration name will be codeautolink_warn_on_unassigned_name, if we decide to do this.
This would require us to change our parsing logic a bit. It's not like we currently have with builtins, which we can fetch from the builtins module. But we could use similar dummy components to gather the warnings and filter them before passing transformations onwards. Not too hard, but we still need to decide if we want this or not.

I'm leaning on this not being a thing, because codeautolink has its use case. This feels more like docs linting, which could be another package altogether. And if they want to use our directives to control how the code behaves, that's all good.


Define Purpose/Plan

What is the benefit this will have over existing https://try.github.io/ and https://guides.github.com/ ?
Is this planned to be more specific to the tools used within UCD? Or a PDF introduction?

@aakash1104 If you had any particular plans for this would be great to hear those! I can start working on this at some point.

@matthewsot This would ideally be an intro git tutorial for students. I would ideally like to have something with more visual representation. We don't want to have complex git commands and confuse students with git cheatsheets -- those already exist. What we do want however, is "How to create a repo?", "add your first file", "commit it", "push it", "see github -- it's there", "How to create branches, merge"
But everything should be done using screenshots so that students can compare and actually see for themselves what's going on. We should then talk about pull requests, and other things and add screenshots as well.
My end goal for this is to have this tutorial be the only tutorial that students at UCD should do. If we're feeling brave, we should write our own auto grader for a simple git tutorial.
I spoke to @joel-porquet during the summer about this and it was actually one of his suggestions, and I'll probably talk to him about having an auto grader for a git tutorial (if something like that is possible) sometime during next quarter. He has students use git extensively in his ECS 150 (for project maintenance). So it'll be very useful for students to get an extra resource!
Ideally, I'd like to have an intro git tutorial by the end of this break.

That sounds really great! Worth noting that https://try.github.io/ has a couple of rather interactive things and https://guides.github.com/ has some nice animations, so could be good inspiration.
I think that we could definitely have some kind of interactive auto-grader -- maybe a custom git server where students clone a repository, then have to push some changes up, then afterwards have to pull changes to get the instructions for the next step, etc. So we have a website that explains how to do the first step (git clone, whatever) then everything else they have to do by actually interacting with Git.

@aakash1104 How well-defined is your conception of this right now? If you have a list of things you would like it to have and would like my help, I can start picking off at that list during the break (or if you'd like brainstorming help of course!).

@matthewsot here's a list of things that I feel should be covered in the git tutorial:

In very simple (non technical) words, what is git and why do we need it?
How to create a repository
Writing your first file
Adding, Committing, and Pushing to git.
Modifying files.
Collaborating  with multiple people
Branches
Pulling changes
Stashing Changes
Tags
Pull Requests
Creating bundles

Like you mentioned earlier, I think we could have some auto grader for most of these topics.
If you think I've missed something, let me know! :)

OK, it's a little bit GitHub-centric, but how would that compare to https://lab.github.com/ ? Specifically, they have the following interactive "courses:" https://lab.github.com/githubtraining/introduction-to-github https://lab.github.com/githubtraining/reviewing-pull-requests https://lab.github.com/githubtraining/managing-merge-conflicts that seem very relevant, even if they don't cover everything mentioned.
If those don't fit the bill, can I clarify if we should focus this specifically on GitHub, or just git itself? I think that would change how we approach the autograder (a modified git server vs. a GitHub "bot").
If we want to focus on GitHub, it seems like we would basically be putting together what they have on lab.github.com -- a website some simple explanations/tutorial that you follow along with in real time while a bot acts as your collaborator and grader. Except maybe tailored to a few different topics (I don't know if they cover bundles or stashing).
If we want to focus on just Git, we would probably need to set up a Git server for them to work on (or set up something within CSIF), but other than that the basic approach seems to be the same -- an online tutorial with some sort of bot that acts as the collaborator & grader.


Live example not working

http://jieter.github.io/Leaflet.layerscontrol-minimap/ yields HTTP 404.

I can reproduce.


CI improvements

As of 2024-01, the waiting time was ~15 minutes which makes it more difficult to land changes. We should reduce the runtime of CI jobs as much as possible.

[ ] CI jobs time is under 5 minutes (6 minutes now)
[ ] CI is not flaky

I think we are good with the latest changes, if we want to go under 5 min we need to update the shards number.
For flakiness we should track individual test rather then one place.


When will LuxCore V2.5 be released?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

There are a couple of open issues still under investigation.  The rc1 is available (and daily builds too) so it is already accessible and we are not in particularly hurry, should we ?

Thanks for your reply. We'll use the rc1 version for now.


fails 2 tests on 32bit archs

I have packaged rusty_pool for Debian, but experience the testsuite failing for the architectures armv5te armv7 i686 mipsel - basically all of the 32bit architectures supported by Debian (overview of build logs here).
Below are details for armv5te (the other build failures look similar from a quick look):
---- tests::test_try_increment_worker_total stdout ----
thread 'tests::test_try_increment_worker_total' panicked at 'assertion failed: `(left == right)`
  left: `65536`,
 right: `0`', src/lib.rs:1802:9
stack backtrace:
   0: rust_begin_unwind
             at /usr/src/rustc-1.61.0/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /usr/src/rustc-1.61.0/library/core/src/panicking.rs:143:14
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed
             at /usr/src/rustc-1.61.0/library/core/src/panicking.rs:182:5
   4: rusty_pool::tests::test_try_increment_worker_total
   5: core::ops::function::FnOnce::call_once
             at /usr/src/rustc-1.61.0/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

---- tests::worker_count_test stdout ----
thread 'tests::worker_count_test' panicked at 'assertion failed: `(left == right)`
  left: `63581`,
 right: `456797`', src/lib.rs:1766:9
stack backtrace:
   0: rust_begin_unwind
             at /usr/src/rustc-1.61.0/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /usr/src/rustc-1.61.0/library/core/src/panicking.rs:143:14
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed
             at /usr/src/rustc-1.61.0/library/core/src/panicking.rs:182:5
   4: rusty_pool::tests::worker_count_test
   5: core::ops::function::FnOnce::call_once
             at /usr/src/rustc-1.61.0/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Please tell if I can do more to help shed light on this - e.g. if helpful that I rebuild with "BACKTRACE=full".

The pool uses one AtomicUsize to store both the number of idle threads and total threads, on 32 bit platforms that means 16 bit each.
For test_try_increment_worker_total the problem is right here:
assert_eq!(witness, 0x0000_0001_0000_0000);
This was probably simply forgotten when moving from AtomicU64 to AtomicUsize and should be changed to something like 1 << (std::mem::size_of::<usize>() * 8 / 2) to work on all platforms.
The test worker_count_test simply overflows on 32 bit because it reaches numbers above 2^16 - 1. Luckily this isn't actually a problem in the pool because the builder validates the upper bounds of the max pool size. The test should either just use lower numbers or check something like #[cfg(target_pointer_width = "32")].


vertical output for preview

This PR is added vertical output for preview
(like the MySQL ¥G)
Is this OK?
example output:
$ java -jar ./embulk-0.4.6.jar preview --output-style vertical config.yml
******************* 1 **********************
            id:long: 1
       account:long: 32,864
     time:timestamp: 2015-01-27 19:23:49 UTC
 purchase:timestamp: 2015-01-27 00:00:00 UTC
     comment:string: embulk

******************* 2 **********************
            id:long: 2
       account:long: 14,824
     time:timestamp: 2015-01-27 19:01:23 UTC
 purchase:timestamp: 2015-01-27 00:00:00 UTC
     comment:string: embulk jruby

******************* 3 **********************
            id:long: 3
       account:long: 27,559
     time:timestamp: 2015-01-28 02:20:02 UTC
 purchase:timestamp: 2015-01-28 00:00:00 UTC
     comment:string: Embulk "csv" parser plugin

Good idea! Thank you.
Handling of "sample" seems wrong because the implementation is originally broken :-(
I followed up at #100.

Thank you for follow up!


Precise and clear instructions / way of including TdLib in a Tizen Project

Part of the answer is in https://github.com/tdlib/td/issues/908#issuecomment-801237582 but nothing is very precise and obvious.
Extract from TdLib README :

Cross-platform: TDLib can be used on Android, iOS, Windows, macOS, Linux, FreeBSD, OpenBSD, NetBSD, Windows Phone, WebAssembly, watchOS, tvOS, Tizen, Cygwin. It should also work on other *nix systems with or without minimal effort.

Well here for Tizen this is clearly not the case. I went through a lot of hassles and I still don't have a working TdLib to use for my Project. Could someone please explain clearly how to port TdLib to Tizen (given the fact that building on Tizen Studio does not support CMakeLists)

This is fully described in https://github.com/tdlib/td/issues/908#issuecomment-580769345.
There are no exact build instructions, because it is hard to build anything for Tizen without a lot of manual interactions with Tizen Studio like project creation. Built shared library can be added to any Tizen project and used in it.

This is fully described in https://github.com/tdlib/td/issues/908#issuecomment-580769345.
There are no exact build instructions, because it is hard to build anything for Tizen without a lot of manual interactions with Tizen Studio like project creation. Built shared library can be added to any Tizen project and used in it.

And how can a built shared library can be included/added to a project ??

And how can a built shared library can be included/added to a project ??

This is not specific to TDLib or related in any way to TDLib itself. You need to search for the answer in Tizen manuals. For example, for a C++ project you can do something similar to https://developer.tizen.org/community/code-snippet/native-code-snippet/how-make-shared-libary-and-link-it.


Alert creation provider/API inconsistency

Hello,
I have a problem to create alerts. My experience is that the process is very inconsistent.
Basically I define a single notification endpoint (Slack) and single alert.
resource "logzio_endpoint" "this" {
  title         = "tf-managed-slack-notification-endpoint"
  description   = "[TF Managed] Notifications from logz.io"
  endpoint_type = "Slack"
  slack {
    url = var.slack_url
  }
}

resource "logzio_alert" "aws_sso" {
  title                    = "[TF] Whatever"
  description              = "Whatever"
  query_string             = "type:cloudtrail AND recipientAccountId:123456789012 AND type:cloudtrail AND eventName:DeleteUser"
  operation                = "GREATER_THAN"
  search_timeframe_minutes = 60
  notification_emails      = []
  value_aggregation_type   = "NONE"
  alert_notification_endpoints   = [logzio_endpoint.this.id]
  suppress_notifications_minutes = 5
  is_enabled                     = true
  severity_threshold_tiers {
    severity  = "MEDIUM"
    threshold = 0
  }
  tags = ["aws-sso", "terraform-managed"]
}

Now some runs are successful and some are not.
A successful one looks like this:
u@localhost:~/logzio-example-alarm$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding logzio/logzio versions matching "1.1.8"...
- Installing logzio/logzio v1.1.8...
- Installed logzio/logzio v1.1.8 (signed by a HashiCorp partner, key ID AF14891A31BE4760)
// ...

u@localhost:~/logzio-example-alarm$ terraform apply -var api_token=my-redacted-token -var slack_url="my-redacted-slack-url"

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # logzio_alert.aws_sso will be created
  + resource "logzio_alert" "aws_sso" {
      + alert_notification_endpoints   = (known after apply)
      + description                    = "Whatever"
      + filter                         = jsonencode(
            {
              + bool = {
                  + filter   = []
                  + must     = []
                  + must_not = []
                  + should   = []
                }
            }
        )
      + id                             = (known after apply)
      + is_enabled                     = true
      + notification_emails            = []
      + operation                      = "GREATER_THAN"
      + query_string                   = "type:cloudtrail AND recipientAccountId:123456789012 AND type:cloudtrail AND eventName:DeleteUser"
      + search_timeframe_minutes       = 60
      + suppress_notifications_minutes = 5
      + tags                           = [
          + "aws-sso",
          + "terraform-managed",
        ]
      + title                          = "[TF] Whatever"
      + value_aggregation_type         = "NONE"

      + severity_threshold_tiers {
          + severity  = "MEDIUM"
          + threshold = 0
        }
    }

  # logzio_endpoint.this will be created
  + resource "logzio_endpoint" "this" {
      + description   = "[TF Managed] Notifications from logz.io"
      + endpoint_type = "Slack"
      + id            = (known after apply)
      + title         = "tf-managed-slack-notification-endpoint"

      + slack {
          + url = "my-redacted-slack-url"
        }
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

logzio_endpoint.this: Creating...
logzio_endpoint.this: Creation complete after 1s [id=36005]
logzio_alert.aws_sso: Creating...
logzio_alert.aws_sso: Creation complete after 1s [id=1215900]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Unfortunately, majority of times it looks more like this which an example of a failure:
u@localhost:~/logzio-example-alarm$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding logzio/logzio versions matching "1.1.8"...
- Installing logzio/logzio v1.1.8...
- Installed logzio/logzio v1.1.8 (signed by a HashiCorp partner, key ID AF14891A31BE4760)

//...

u@localhost:~/logzio-example-alarm$ terraform apply -var api_token=my-redacted-token -var slack_url="my-redacted-slack-url"

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # logzio_alert.aws_sso will be created
  + resource "logzio_alert" "aws_sso" {
      + alert_notification_endpoints   = (known after apply)
      + description                    = "Whatever"
      + filter                         = jsonencode(
            {
              + bool = {
                  + filter   = []
                  + must     = []
                  + must_not = []
                  + should   = []
                }
            }
        )
      + id                             = (known after apply)
      + is_enabled                     = true
      + notification_emails            = []
      + operation                      = "GREATER_THAN"
      + query_string                   = "type:cloudtrail AND recipientAccountId:123456789012 AND type:cloudtrail AND eventName:DeleteUser"
      + search_timeframe_minutes       = 60
      + suppress_notifications_minutes = 5
      + tags                           = [
          + "aws-sso",
          + "terraform-managed",
        ]
      + title                          = "[TF] Whatever"
      + value_aggregation_type         = "NONE"

      + severity_threshold_tiers {
          + severity  = "MEDIUM"
          + threshold = 0
        }
    }

  # logzio_endpoint.this will be created
  + resource "logzio_endpoint" "this" {
      + description   = "[TF Managed] Notifications from logz.io"
      + endpoint_type = "Slack"
      + id            = (known after apply)
      + title         = "tf-managed-slack-notification-endpoint"

      + slack {
          + url = "my-redacted-slack-url"
        }
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

logzio_endpoint.this: Creating...
logzio_endpoint.this: Creation complete after 1s [id=36007]
logzio_alert.aws_sso: Creating...

Error: API call GetAlert failed with status code 404, data: "no alert id 1215907 found for account 123456"

  on main.tf line 10, in resource "logzio_alert" "aws_sso":
  10: resource "logzio_alert" "aws_sso" {

u@localhost:~/logzio-example-alarm$
u@localhost:~/logzio-example-alarm$
u@localhost:~/logzio-example-alarm$ terraform apply -var api_token=my-redacted-token -var slack_url="my-redacted-slack-url"
logzio_endpoint.this: Refreshing state... [id=36007]
logzio_alert.aws_sso: Refreshing state... [id=1215907]

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # logzio_alert.aws_sso is tainted, so must be replaced
-/+ resource "logzio_alert" "aws_sso" {
        alert_notification_endpoints   = [
            36007,
        ]
        description                    = "Whatever"
        filter                         = jsonencode(
            {
                bool = {
                    filter   = []
                    must     = []
                    must_not = []
                    should   = []
                }
            }
        )
      - group_by_aggregation_fields    = [] -> null
      ~ id                             = "1215907" -> (known after apply)
        is_enabled                     = true
        notification_emails            = []
        operation                      = "GREATER_THAN"
        query_string                   = "type:cloudtrail AND recipientAccountId:123456789012 AND type:cloudtrail AND eventName:DeleteUser"
        search_timeframe_minutes       = 60
        suppress_notifications_minutes = 5
        tags                           = [
            "aws-sso",
            "terraform-managed",
        ]
        title                          = "[TF] Whatever"
        value_aggregation_type         = "NONE"

        severity_threshold_tiers {
            severity  = "MEDIUM"
            threshold = 0
        }
    }

Plan: 1 to add, 0 to change, 1 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

logzio_alert.aws_sso: Destroying... [id=1215907]
logzio_alert.aws_sso: Destruction complete after 1s
logzio_alert.aws_sso: Creating...

Error: API call GetAlert failed with status code 404, data: "no alert id 1215909 found for account 123456"

  on main.tf line 10, in resource "logzio_alert" "aws_sso":
  10: resource "logzio_alert" "aws_sso" {

When I go to web console I can see the alarm actually created (after first FAILED apply) with the id that's supposedly not found by logzio tf provider.
My setup
terraform version
Terraform v0.13.6 // Although testes with 0.14.10, too
+ provider registry.terraform.io/logzio/logzio v1.1.8

Hi @Veluu ,
This is a known issue and we are currently working on a fix 🙂

@Veluu
Our new provider version (v1.2.0) should fix this issue, please let us know if you're having further issues with it.
(#69)

Yeah, the 4s delay fixed the problem.


3-gpu-problems section some text modifications

The part related to Vasp in this section is not complete, it seems that some "simple cases" are expected

thanks Pedro. I plan to add some simple timing for e.g. matrix multiplication on CPU vs GPU to showcase a best-case scenario


net: sockets: Do not start service thread if too little resources

If the CONFIG_NET_SOCKETS_POLL_MAX is smaller than what is needed for the socket service API to work properly, then we should not start the service thread as the service API cannot work and might cause memory overwrite in ctx.events[] array.
Fixes #69233

This PR fixes a medium priority bug, but only blocker bug fixes can be merged after -rc3.

I suggested medium, but of course can increase to high if needed.