Release (2021-05-11)

:tada: This PR is included in version 0.14.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


My go at a Docker image

This pull request...

[ ] Fixes a bug
[x] Introduces a new feature
[ ] Improves an existing feature
[ ] Boosts code quality or performance

Description
I realize I'm not the first one to submit a Docker PR, but I'd still like to post how I've set it up.
Configuration
The configuration can be set in multiple ways with this image:

Firstly, a config.txt file can be added in /bot-data/ or /config/ (through a volume mount or by extending from this image and adding the file to /config/). This file will be copied to the bot's directory.
Secondly, config values can be set through environment variables. This allows one to change the settings without a config.txt file. It also means it's possible to e.g. get the bot token from a secret in Kubernetes. Any environment variable prefixed with MB- will be added to the configuration (without the prefix). So, for example, to set the token and owner, set the following environment variables:

MB-token=abc123
MB-owner=12345

Volume mounts
The serversettings.json file and playlists folder are saved to the /bot-data/ directory. It's recommended to mount this folder to some persistent storage, or the data is gonna be lost when the container is deleted. It's possible to override the playlists folder in the config, but not recommended to do so. A config.txt file can also be added to the /bot-data/ folder in order to set the configuration. It's also possible to add the config.txt file to /config/ instead, which would also require a mount.
Building the image
Right now, the Dockerfile expects there to be a JMusicBot-<version>.jar file in the docker folder for it to build. The version needs to be set using the VERSION build argument. Of course, this can be changed as needed to make it more convenient to build the image. I haven't put too much thought into this, because I'm not very familiar with Java and how to build the bot.
Basic instructions to build the image (open for improvement):

Copy the JMusicBot-<version>.jar file to the docker folder
Run docker build -t musicbot:latest --build-arg VERSION=0.3.5 . (change version as needed)
To start, run: docker run -e MB-token=abc123 -e MB-owner=12345 musicbot (set the token and owner correctly)

The start example doesn't include a volume mount for /bot-data/, it's probably best to add that.
Kubernetes deployment example

An example of a Kubernetes deployment I used to run the bot
Note that the image name depends on where the image is hosted and the volume mount depends on the Kubernetes hosting situation.
apiVersion: apps/v1
kind: Deployment
metadata:
  name: musicbot
  namespace: musicbot
  labels:
    app: musicbot
spec:
  replicas: 1
  selector:
    matchLabels:
      app: musicbot
  template:
    metadata:
      labels:
        app: musicbot
    spec:
      containers:
      - name: musicbot
        image: musicbot:latest
        env:
        - name: MB-token
          value: abc123
        - name: MB-owner
          value: '12345'
        - name: MB-game
          value: '"Listening to music"'
        - name: MB-alonetimeuntilstop
          value: '60'
        volumeMounts:
          - name: musicbot-data
            mountPath: /bot-data
        resources:
          requests:
            memory: 64Mi
            cpu: 125m
          limits:
            memory: 512Mi
            cpu: 500m
      volumes:
      - name: musicbot-data
        [...]

https://github.com/alshdavid/j-music-bot


Unexpected behavior with [un]installing global pkgs

I found that with the new separate jpm command, I cannot uninstall global packages. For some,  spork, for example, the solution is to install the package again and then uninstall it. For some others, miniz, for example, this does not work either.

I can reproduce this behavior on another system as well. There I tried to remove /usr/local/lib/janet and fresh install janet and jpm from source with the same result. I will try to investigate more later and come back with what I have found.

So only culprit I was able to find with my limited knowledge of the whole system is that native modules are found in module/cache at https://github.com/janet-lang/jpm/blob/cf6086ae3b282046141fd7b6fbd1d08edcb85fdd/jpm/cc.janet#L284 and that the cache is populated with the global native modules only.

FWIW, I tried with both spork and janet-miniz and here (Manjaro Linux) it seemed to work ok.
I don't have things installed under /usr/local though.
I can try in a VM for a /usr/local installation.

I didn't notice any difference -- it was with a VM running Manjaro Linux though.
May be it depends on the distribution?

The second systém i have used is also Manjaro. I guess it is something with my systems installation being old. I will try to clean it up and do fresh install thank you for testing @sogaiu. I am closing this as it seems to be user error.


Web - The "Edit message" option is not displayed on sent messages after changing the default contact method until reloading the report

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:
PRECONDITION: Account A has some contact methods added

Log in as Account A
Open any chat report that has messages sent by Account A
check that there is an Edit option on each of these messages
Open Profile settings >> select any contact method and “Set as default”
Re-check the chat report step #1
Hover on these messages to view options

Expected Result:
Edit message option should still be displayed and there is NO  “Flag as offensive” option
Actual Result:
The Edit message option is NOT displayed until RELOAD the report
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[x] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: 1.3.34-1
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/93399543/2d6efe84-b473-47df-a45b-507b8eefcecf
https://github.com/Expensify/App/assets/93399543/821f917c-ebad-4464-9496-de7158f42324
Expensify/Expensify Issue URL:
Issue reported by:  @tranvantoan-qn
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1688325484455269
View all open jobs on GitHub

@kushu7
Yeah, It looks like so,
And I'm sorry if it's not appropriate to ask. But since I can provide more information which is "until reloading the report" - it may help fix the issue easier and more comprehensively proposals.  Can I still can get a part of bug bounty?

Yes, this is a dupe, closing in favor of https://github.com/Expensify/App/issues/21948.
@tranvantoan-qn it's fine to ask! Right now, we're only offering a bug reporting bonus to a single reporter for each issue. We expect bug reports to be complete, so while it's sometimes helpful when others add additional details, we can't compensate for that.


Resize Images messes up animated gifs

Microsoft PowerToys version
0.41.4
Running as admin

[ ] Yes

Area(s) with issue?
Image Resizer
Steps to reproduce
Take any animated gif and resize it, most likely the file size will increase 10 fold and the animation killed. Or, the images are blended together and a total mess is saved. Should either exclude animated gifs from resizing or implement a correct algorithm.

✔ Expected Behavior
Correctly resized animated gif with all settings retained.
❌ Actual Behavior
Depending on the type of gif, the resulting file is always much bigger and animations are either lost or mixed together.
Other Software
No response

@jaimecbernardo we should investigate this in v0.45 to see if a clean solution is possible. If not, let's just restrict GIF's from being resized by v0.47.

@dedavis6797 ,
Assigning myself to it to verify what's the root cause.

I believe these issues are caused by the limitations of the .net GifBitmapEncoder class:
https://docs.microsoft.com/en-us/dotnet/api/system.windows.media.imaging.gifbitmapencoder?view=net-5.0#remarks

Graphics Interchange Format (GIF) images do not support global preview, global thumbnails, global metadata, frame level thumbnails, or frame level metadata. Graphics Interchange Format (GIF) images do support multiple frames.

This description is not quite accurate. GIF images do support those, but the .net encoder doesn't. For the case of the images that are submitted by the user, they seem to use global metadata (for looping) and frame level metadata (offset,delay) at least.
The way to fix this would be not use the .net encoder and replace it with an encoder that supports more GIF features. This would be a bigger item, including evaluating, testing and integrating other open source libraries to replace GifBitmapEncoder.
I don't think this warrants disabling the feature altogether, since it would still work for gif files that don't use these features. Perhaps a warning in the Image Resizer UI that some gif features are not supported, when trying to resize gif files, would be the way to go. What do you think?
@dedavis6797
@Hefaistos68

I've created the suggestion to replace the gif encoder. In the meanwhile, a Warning will be added while resizing gif images.

I don't think this warrants disabling the feature altogether, since it would still work for gif files that don't use these features. Perhaps a warning in the Image Resizer UI that some gif features are not supported, when trying to resize gif files, would be the way to go. What do you think?

I agree, that warning should be fine for now given the large cost of updating the encoder. If adding the warning is low cost, lets target having that in for v0.45 so users are aware.
And thank you for creating #12617, we can set a timeline for that work item following the completion of existing P0 and P1 items.

Resolved with the release of v0.45.


RB Utils file

Summary
This PR imports the rb_utils file from qiskit ignis, in the following version: https://github.com/Qiskit/qiskit-ignis/blob/81955597ebabe1870088e4adc606f7ad2d71ffa8/qiskit/ignis/verification/randomized_benchmarking/rb_utils.py
Details and comments
The rb_utils file contains the auxiliary methods required for conversion between EPC (error per Clifford) obtained directly from RB, and EPG (error per gate in the gateset basis), the goal of RB.
When importing the file, two major changes were done:

Deprecated methods and parts of the code were removed
The gateset was changed from [u1, u2, u3, cx] to [rz, x, sx, cx]

How these functions are related to the RBAnalysis class? How should the outputs be stored in the ResultsDB?
I would suggest to make a class for all these function. The function gates_per_clifford can be an internal function _gates_per_clifford, since it's a helper function, while the other functions are externalized.
It would be good to add these functions to the RB tutorial notebook (and compare their outcome with the backend gate errors)
How should the coherence_limit function interact with the other experiments, like the T1 experiment?
The basis gates [rz, sx, x, cx] are hard-coded in the functions. Shouldn't they be only a default option?
Do we want the user to be able to define other basis gates (and then also provide the ratios of their errors if needed)?

Replaced by a more extensive PR, #86


Fix RunRouter subfactory bug

This PR addresses #170.
Description

Motivation and Context

How Has This Been Tested?

The initial commit adds a test to demonstrate the bug, but the test may not be quite right yet. At least it fails!

An elegant test.
I agree we are overdue for splitting up the event-model tests across more than more file. Would you mind moving this embarrassingly monolithic "unit test" into the new file as part of this PR, so we can keep the RunRouter tests together?

I confirmed that running the test in its final (fixed) form against the master branch version of event-model, I can reproduce the expected bug:
>       assert subfactory_documents["start"] == [run_bundle.start_doc]
E       AssertionError: assert [{'configurat...rimary', ...}] == [{'time': 158...742a544daa0'}]


ci: Added markdown-link-check.yml

Checklist before requesting a review

[x] I have performed a self-review of my code.
[x] Implemented tests for new features.
[x] I tested the features already implemented.
[x] Added usage examples in the readme.

:tada: This PR is included in version 1.4.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


bug: don't trigger on file delete

Hi,
I'm using npm-watch and I don't know why it doesn't trigger when a file is deleted.
any idea ?
https://github.com/Talend/ui/pull/115

find it here: https://github.com/remy/nodemon/issues/656


Update terraform requirements

Bumped all the terraform providers to the latest versions & relocked.
Also bumped the esp-v2 image to the latest.

Just wondering if you want to stagger the update to staging versus production by a week to give it soak time in staging?

Yeah, I think I'll merge this after this week's release.


Add hdinsight-spark-linux example

I see a potential issue in this template. I use uniqueString(resourceGroup().idto generate the name of the storage account.
Afterwards, the same function is use at multiple places to reference the storage account. However, if I understand well the function, it generates a new output each time you call it. So the reference will fail.
What do you think @alex-frankel?


Change Program to Team in notification messages

Hey @markwooff ,
When I publish and unpublish a team I get the following messages:  (to change to 'Team' pls).  Note:  I'm adding these to my list of things to get translated. :)

This has been fixed. Thanks for finding these additional texts where translation is needed!


failed to build pyarrow during installation of Optimus

I have Python 3.7.2 installed. While executing pip3 install optimuspyspark I get the following result
Collecting optimuspyspark
Using cached https://files.pythonhosted.org/packages/5e/2d/5e4cac6f648b3474dc40571c34219b9d20948087c38a157ebf559fff597a/optimuspyspark-2.1.5-py3-none-any.whl
Collecting pymongo==3.7.2 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/99/18/b50834fbfd557eaf07985c2a657b03efc691462ecba62d03e32c2fc4f640/pymongo-3.7.2-cp37-cp37m-manylinux1_x86_64.whl
Collecting ratelimit==2.2.1 (from optimuspyspark)
Collecting matplotlib==3.0.1 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/ab/e9/675be8ef5ca1be676c5a5b2dbb3eb5246c7017eaa7b233638562a61ea968/matplotlib-3.0.1-cp37-cp37m-manylinux1_x86_64.whl
Collecting pyspark==2.4.0 (from optimuspyspark)
Requirement already satisfied: tensorflow==1.12.0 in ./cleaning/lib/python3.7/site-packages (from optimuspyspark) (1.12.0)
Collecting seaborn==0.9.0 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/a8/76/220ba4420459d9c4c9c9587c6ce607bf56c25b3d3d2de62056efe482dadc/seaborn-0.9.0-py3-none-any.whl
Requirement already satisfied: numpy==1.15.2 in ./cleaning/lib/python3.7/site-packages (from optimuspyspark) (1.15.2)
Requirement already satisfied: h5py>=2.7.0 in ./cleaning/lib/python3.7/site-packages (from optimuspyspark) (2.9.0)
Collecting backoff==1.8.0 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/00/b9/b045f0fe02aa80cefc5a6921d5f7674db58c1658d0e4b888562c15ef6aba/backoff-1.8.0-py2.py3-none-any.whl
Collecting humanize==0.5.1 (from optimuspyspark)
Collecting h2o-pysparkling-2.4==2.4.4 (from optimuspyspark)
Requirement already satisfied: deprecated==1.2.4 in ./cleaning/lib/python3.7/site-packages (from optimuspyspark) (1.2.4)
Requirement already satisfied: setuptools==40.6.3 in ./cleaning/lib/python3.7/site-packages (from optimuspyspark) (40.6.3)
Collecting ipython==7.2.0 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/f0/b4/a9ea018c73a84ee6280b2e94a1a6af8d63e45903eac2da0640fa63bca4db/ipython-7.2.0-py3-none-any.whl
Collecting nose==1.3.7 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/15/d8/dd071918c040f50fa1cf80da16423af51ff8ce4a0f2399b7bf8de45ac3d9/nose-1.3.7-py3-none-any.whl
Requirement already satisfied: fastnumbers==2.1.1 in ./cleaning/lib/python3.7/site-packages (from optimuspyspark) (2.1.1)
Collecting pika==0.12.0 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/bf/48/72de47f63ba353bacd74b76bb65bc63620b0706d8b0471798087cd5a4916/pika-0.12.0-py2.py3-none-any.whl
Collecting pygments>=2.2.0 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/13/e5/6d710c9cf96c31ac82657bcfb441df328b22df8564d58d0c4cd62612674c/Pygments-2.3.1-py2.py3-none-any.whl
Collecting findspark==1.3.0 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/b1/c8/e6e1f6a303ae5122dc28d131b5a67c5eb87cbf8f7ac5b9f87764ea1b1e1e/findspark-1.3.0-py2.py3-none-any.whl
Requirement already satisfied: six>=1.10.0 in ./cleaning/lib/python3.7/site-packages (from optimuspyspark) (1.12.0)
Collecting flask==1.0.2 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/7f/e7/08578774ed4536d3242b14dacb4696386634607af824ea997202cd0edb4b/Flask-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: requests==2.21.0 in ./cleaning/lib/python3.7/site-packages (from optimuspyspark) (2.21.0)
Requirement already satisfied: tqdm==4.28.1 in ./cleaning/lib/python3.7/site-packages (from optimuspyspark) (4.28.1)
Collecting Jinja2==2.10 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl
Collecting pandas==0.23.4 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/67/a7/12261a51ac2e7be4c698ca27cbe364ca5f16d64999456ee47ea8c7b44417/pandas-0.23.4-cp37-cp37m-manylinux1_x86_64.whl
Collecting pillow==5.3.0 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/62/8c/230204b8e968f6db00c765624f51cfd1ecb6aea57b25ba00b240ee3fb0bd/Pillow-5.3.0-cp37-cp37m-manylinux1_x86_64.whl
Collecting tabulate==0.8.2 (from optimuspyspark)
Requirement already satisfied: psutil==5.4.8 in ./cleaning/lib/python3.7/site-packages (from optimuspyspark) (5.4.8)
Collecting multipledispatch==0.6.0 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/89/79/429ecef45fd5e4504f7474d4c3c3c4668c267be3370e4c2fd33e61506833/multipledispatch-0.6.0-py3-none-any.whl
Collecting pyarrow==0.10.0 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/c0/a0/f7e9dfd8988d94f4952f9b50eb04e14a80fbe39218520725aab53daab57c/pyarrow-0.10.0.tar.gz
Collecting keras==2.2.4 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/5e/10/aa32dad071ce52b5502266b5c659451cfd6ffcbf14e6c8c4f16c0ff5aaab/Keras-2.2.4-py2.py3-none-any.whl
Collecting kombu==4.2.2.post1 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/44/5d/a4162de3033be4b42e59ee5101de92b218a6b0a943de9cbe8bab5c45479a/kombu-4.2.2.post1-py2.py3-none-any.whl
Collecting python-dateutil==2.7.5 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/74/68/d87d9b36af36f44254a8d512cbfc48369103a3b9e474be9bdfe536abfc45/python_dateutil-2.7.5-py2.py3-none-any.whl
Collecting pytest==4.0.1 (from optimuspyspark)
Using cached https://files.pythonhosted.org/packages/81/27/d4302e4e00497448081120f65029696070806bc8e649b83f644de006d710/pytest-4.0.1-py2.py3-none-any.whl
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 (from matplotlib==3.0.1->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/de/0a/001be530836743d8be6c2d85069f46fecf84ac6c18c7f5fb8125ee11d854/pyparsing-2.3.1-py2.py3-none-any.whl
Collecting kiwisolver>=1.0.1 (from matplotlib==3.0.1->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/5c/7e/d6cae2f241ba474a2665f24b480bf4e247036d63939dda2bbc4d2ee5069d/kiwisolver-1.0.1-cp37-cp37m-manylinux1_x86_64.whl
Collecting cycler>=0.10 (from matplotlib==3.0.1->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/f7/d2/e07d3ebb2bd7af696440ce7e754c59dd546ffe1bbe732c8ab68b9c834e61/cycler-0.10.0-py2.py3-none-any.whl
Collecting py4j==0.10.7 (from pyspark==2.4.0->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/e3/53/c737818eb9a7dc32a7cd4f1396e787bd94200c3997c72c1dbe028587bd76/py4j-0.10.7-py2.py3-none-any.whl
Requirement already satisfied: gast>=0.2.0 in ./cleaning/lib/python3.7/site-packages (from tensorflow==1.12.0->optimuspyspark) (0.2.2)
Requirement already satisfied: absl-py>=0.1.6 in ./cleaning/lib/python3.7/site-packages (from tensorflow==1.12.0->optimuspyspark) (0.7.0)
Requirement already satisfied: keras-preprocessing>=1.0.5 in ./cleaning/lib/python3.7/site-packages (from tensorflow==1.12.0->optimuspyspark) (1.0.8)
Requirement already satisfied: wheel>=0.26 in ./cleaning/lib/python3.7/site-packages (from tensorflow==1.12.0->optimuspyspark) (0.32.3)
Requirement already satisfied: tensorboard<1.13.0,>=1.12.0 in ./cleaning/lib/python3.7/site-packages (from tensorflow==1.12.0->optimuspyspark) (1.12.2)
Requirement already satisfied: keras-applications>=1.0.6 in ./cleaning/lib/python3.7/site-packages (from tensorflow==1.12.0->optimuspyspark) (1.0.7)
Requirement already satisfied: astor>=0.6.0 in ./cleaning/lib/python3.7/site-packages (from tensorflow==1.12.0->optimuspyspark) (0.7.1)
Requirement already satisfied: termcolor>=1.1.0 in ./cleaning/lib/python3.7/site-packages (from tensorflow==1.12.0->optimuspyspark) (1.1.0)
Requirement already satisfied: grpcio>=1.8.6 in ./cleaning/lib/python3.7/site-packages (from tensorflow==1.12.0->optimuspyspark) (1.18.0)
Requirement already satisfied: protobuf>=3.6.1 in ./cleaning/lib/python3.7/site-packages (from tensorflow==1.12.0->optimuspyspark) (3.6.1)
Collecting scipy>=0.14.0 (from seaborn==0.9.0->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/80/39/066ecde98f373430bf7a39a02d91c7075b01ef4fc928456e8e31577342d6/scipy-1.2.0-cp37-cp37m-manylinux1_x86_64.whl
Collecting future (from h2o-pysparkling-2.4==2.4.4->optimuspyspark)
Collecting colorama>=0.3.8 (from h2o-pysparkling-2.4==2.4.4->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Requirement already satisfied: wrapt<2,>=1 in ./cleaning/lib/python3.7/site-packages (from deprecated==1.2.4->optimuspyspark) (1.11.1)
Collecting traitlets>=4.2 (from ipython==7.2.0->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/93/d6/abcb22de61d78e2fc3959c964628a5771e47e7cc60d53e9342e21ed6cc9a/traitlets-4.3.2-py2.py3-none-any.whl
Collecting jedi>=0.10 (from ipython==7.2.0->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/c2/bc/54d53f5bc4658380d0eca9055d72be4df45e5bfd91a4bac97da224a92553/jedi-0.13.2-py2.py3-none-any.whl
Collecting decorator (from ipython==7.2.0->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/f1/cd/7c8240007e9716b14679bc217a1baefa4432aa30394f7e2ec40a52b1a708/decorator-4.3.2-py2.py3-none-any.whl
Collecting prompt-toolkit<2.1.0,>=2.0.0 (from ipython==7.2.0->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/65/c2/e676da701cda11b32ff42eceb44aa7d8934b597d604bb5e94c0283def064/prompt_toolkit-2.0.8-py3-none-any.whl
Collecting backcall (from ipython==7.2.0->optimuspyspark)
Collecting pickleshare (from ipython==7.2.0->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/9a/41/220f49aaea88bc6fa6cba8d05ecf24676326156c23b991e80b3f2fc24c77/pickleshare-0.7.5-py2.py3-none-any.whl
Collecting pexpect; sys_platform != "win32" (from ipython==7.2.0->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/89/e6/b5a1de8b0cc4e07ca1b305a4fcc3f9806025c1b651ea302646341222f88b/pexpect-4.6.0-py2.py3-none-any.whl
Requirement already satisfied: Werkzeug>=0.14 in ./cleaning/lib/python3.7/site-packages (from flask==1.0.2->optimuspyspark) (0.14.1)
Collecting click>=5.1 (from flask==1.0.2->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl
Collecting itsdangerous>=0.24 (from flask==1.0.2->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/76/ae/44b03b253d6fade317f32c24d100b3b35c2239807046a4c953c7b89fa49e/itsdangerous-1.1.0-py2.py3-none-any.whl
Requirement already satisfied: urllib3<1.25,>=1.21.1 in ./cleaning/lib/python3.7/site-packages (from requests==2.21.0->optimuspyspark) (1.24.1)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./cleaning/lib/python3.7/site-packages (from requests==2.21.0->optimuspyspark) (3.0.4)
Requirement already satisfied: idna<2.9,>=2.5 in ./cleaning/lib/python3.7/site-packages (from requests==2.21.0->optimuspyspark) (2.8)
Requirement already satisfied: certifi>=2017.4.17 in ./cleaning/lib/python3.7/site-packages (from requests==2.21.0->optimuspyspark) (2018.11.29)
Collecting MarkupSafe>=0.23 (from Jinja2==2.10->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/e4/c4/adcc2d6f2ac2146cc04e076f14f1006c1de8e1e747fa067668b6573000b8/MarkupSafe-1.1.0-cp37-cp37m-manylinux1_x86_64.whl
Collecting pytz>=2011k (from pandas==0.23.4->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/61/28/1d3920e4d1d50b19bc5d24398a7cd85cc7b9a75a490570d5a30c57622d34/pytz-2018.9-py2.py3-none-any.whl
Collecting pyyaml (from keras==2.2.4->optimuspyspark)
Collecting amqp<3.0,>=2.1.4 (from kombu==4.2.2.post1->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/27/32/5c8a0d355b247446eb73f89c0fa4a22c1832764c0cc9d2bc43b9256d9366/amqp-2.4.1-py2.py3-none-any.whl
Collecting py>=1.5.0 (from pytest==4.0.1->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/3e/c7/3da685ef117d42ac8d71af525208759742dd235f8094221fdaafcd3dba8f/py-1.7.0-py2.py3-none-any.whl
Collecting atomicwrites>=1.0 (from pytest==4.0.1->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/52/90/6155aa926f43f2b2a22b01be7241be3bfd1ceaf7d0b3267213e8127d41f4/atomicwrites-1.3.0-py2.py3-none-any.whl
Collecting attrs>=17.4.0 (from pytest==4.0.1->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/3a/e1/5f9023cc983f1a628a8c2fd051ad19e76ff7b142a0faf329336f9a62a514/attrs-18.2.0-py2.py3-none-any.whl
Collecting pluggy>=0.7 (from pytest==4.0.1->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/2d/60/f58d9e8197f911f9405bf7e02227b43a2acc2c2f1a8cbb1be5ecf6bfd0b8/pluggy-0.8.1-py2.py3-none-any.whl
Collecting more-itertools>=4.0.0 (from pytest==4.0.1->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/a4/a6/42f17d065bda1fac255db13afc94c93dbfb64393eae37c749b4cb0752fc7/more_itertools-5.0.0-py3-none-any.whl
Requirement already satisfied: markdown>=2.6.8 in ./cleaning/lib/python3.7/site-packages (from tensorboard<1.13.0,>=1.12.0->tensorflow==1.12.0->optimuspyspark) (3.0.1)
Collecting ipython-genutils (from traitlets>=4.2->ipython==7.2.0->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/fa/bc/9bd3b5c2b4774d5f33b2d544f1460be9df7df2fe42f352135381c347c69a/ipython_genutils-0.2.0-py2.py3-none-any.whl
Collecting parso>=0.3.0 (from jedi>=0.10->ipython==7.2.0->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/49/f6/5108eb2c490c57d7ca3ac013e03e5c50f195aa1749b17a1fe553d63a37c2/parso-0.3.2-py2.py3-none-any.whl
Collecting wcwidth (from prompt-toolkit<2.1.0,>=2.0.0->ipython==7.2.0->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/7e/9f/526a6947247599b084ee5232e4f9190a38f398d7300d866af3ab571a5bfe/wcwidth-0.1.7-py2.py3-none-any.whl
Collecting ptyprocess>=0.5 (from pexpect; sys_platform != "win32"->ipython==7.2.0->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/d1/29/605c2cc68a9992d18dada28206eeada56ea4bd07a239669da41674648b6f/ptyprocess-0.6.0-py2.py3-none-any.whl
Collecting vine>=1.1.3 (from amqp<3.0,>=2.1.4->kombu==4.2.2.post1->optimuspyspark)
Using cached https://files.pythonhosted.org/packages/62/dd/e30f828a39914626d67876b987d6fc47616b64de680cd0f746fc9c8aab47/vine-1.2.0-py2.py3-none-any.whl
Building wheels for collected packages: pyarrow
Building wheel for pyarrow (setup.py) ... error
Complete output from command /root/Mehraj/cleaning/bin/python3.7 -u -c "import setuptools, tokenize;file='/tmp/pip-install-hj1dj2ty/pyarrow/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /tmp/pip-wheel-05dwl5wd --python-tag cp37:
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/ipc.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/plasma.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/serialization.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/hdfs.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/types.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/parquet.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/util.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/filesystem.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/formatting.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/pandas_compat.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/orc.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/jvm.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/feather.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/compat.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/init.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/benchmark.py -> build/lib.linux-x86_64-3.7/pyarrow
creating build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_types.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_table.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_hdfs.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/conftest.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_io.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_builder.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_parquet.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_feather.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/util.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_misc.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_deprecations.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/pandas_examples.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_plasma_tf_op.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_serialization.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_tensor.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_array.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_scalars.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_convert_builtin.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_plasma.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_schema.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/deserialize_buffer.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_jvm.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_convert_pandas.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_ipc.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_cython.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/init.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
running egg_info
writing pyarrow.egg-info/PKG-INFO
writing dependency_links to pyarrow.egg-info/dependency_links.txt
writing entry points to pyarrow.egg-info/entry_points.txt
writing requirements to pyarrow.egg-info/requires.txt
writing top-level names to pyarrow.egg-info/top_level.txt
reading manifest file 'pyarrow.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '.so' found anywhere in distribution
warning: no previously-included files matching '.pyc' found anywhere in distribution
warning: no previously-included files matching '~' found anywhere in distribution
warning: no previously-included files matching '#' found anywhere in distribution
warning: no previously-included files matching '.git*' found anywhere in distribution
warning: no previously-included files matching '.DS_Store' found anywhere in distribution
writing manifest file 'pyarrow.egg-info/SOURCES.txt'
copying pyarrow/init.pxd -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/_orc.cpp -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/_orc.pxd -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/_orc.pyx -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/_parquet.cpp -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/_parquet.pxd -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/_parquet.pyx -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/_plasma.cpp -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/_plasma.pyx -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/array.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/benchmark.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/builder.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/error.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/feather.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/io-hdfs.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/io.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/ipc.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/lib.cpp -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/lib.pxd -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/lib.pyx -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/lib_api.h -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/memory.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/plasma_store -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/public-api.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/scalar.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/serialization.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/table.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/types.pxi -> build/lib.linux-x86_64-3.7/pyarrow
creating build/lib.linux-x86_64-3.7/pyarrow/include
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/allocator.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/api.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/array.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/buffer.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/builder.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/compare.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/memory_pool.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/pretty_print.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/record_batch.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/status.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/stl.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/table.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/table_builder.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/tensor.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/test-util.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/type.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/type_fwd.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/type_traits.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/visitor.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/visitor_inline.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/adapters
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/adapters/orc
copying pyarrow/include/arrow/adapters/orc/adapter.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/adapters/orc
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/adapters/tensorflow
copying pyarrow/include/arrow/adapters/tensorflow/convert.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/adapters/tensorflow
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/compute
copying pyarrow/include/arrow/compute/api.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/compute
copying pyarrow/include/arrow/compute/context.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/compute
copying pyarrow/include/arrow/compute/kernel.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/compute
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/compute/kernels
copying pyarrow/include/arrow/compute/kernels/cast.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/compute/kernels
copying pyarrow/include/arrow/compute/kernels/hash.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/compute/kernels
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/io
copying pyarrow/include/arrow/io/api.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/io
copying pyarrow/include/arrow/io/buffered.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/io
copying pyarrow/include/arrow/io/file.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/io
copying pyarrow/include/arrow/io/hdfs.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/io
copying pyarrow/include/arrow/io/interfaces.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/io
copying pyarrow/include/arrow/io/memory.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/io
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/ipc
copying pyarrow/include/arrow/ipc/api.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/ipc
copying pyarrow/include/arrow/ipc/dictionary.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/ipc
copying pyarrow/include/arrow/ipc/feather.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/ipc
copying pyarrow/include/arrow/ipc/json.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/ipc
copying pyarrow/include/arrow/ipc/message.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/ipc
copying pyarrow/include/arrow/ipc/reader.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/ipc
copying pyarrow/include/arrow/ipc/writer.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/ipc
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/api.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/arrow_to_pandas.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/arrow_to_python.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/benchmark.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/builtin_convert.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/common.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/config.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/decimal.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/helpers.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/init.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/io.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/iterators.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/numpy_convert.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/numpy_interop.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/numpy_to_arrow.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/platform.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/pyarrow.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/python_to_arrow.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/type_traits.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/bit-stream-utils.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/bit-util.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/bpacking.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/checked_cast.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/compiler-util.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/compression.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/compression_brotli.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/compression_lz4.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/compression_snappy.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/compression_zlib.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/compression_zstd.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/cpu-info.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/decimal.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/hash-util.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/hash.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/io-util.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/key_value_metadata.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/lazy.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/logging.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/macros.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/memory.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/optional.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/parallel.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/recursive_wrapper.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/rle-encoding.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/sse-util.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/stl.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/string.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/thread-pool.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/type_traits.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/variant.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/variant_cast.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/variant_io.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/variant_visitor.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/visibility.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util/variant
copying pyarrow/include/arrow/util/variant/optional.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util/variant
copying pyarrow/include/arrow/util/variant/recursive_wrapper.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util/variant
copying pyarrow/include/arrow/util/variant/variant_cast.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util/variant
copying pyarrow/include/arrow/util/variant/variant_io.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util/variant
copying pyarrow/include/arrow/util/variant/variant_visitor.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util/variant
creating build/lib.linux-x86_64-3.7/pyarrow/includes
copying pyarrow/includes/init.pxd -> build/lib.linux-x86_64-3.7/pyarrow/includes
copying pyarrow/includes/common.pxd -> build/lib.linux-x86_64-3.7/pyarrow/includes
copying pyarrow/includes/libarrow.pxd -> build/lib.linux-x86_64-3.7/pyarrow/includes
creating build/lib.linux-x86_64-3.7/pyarrow/tensorflow
copying pyarrow/tensorflow/build.sh -> build/lib.linux-x86_64-3.7/pyarrow/tensorflow
copying pyarrow/tensorflow/plasma_op.cc -> build/lib.linux-x86_64-3.7/pyarrow/tensorflow
copying pyarrow/tests/pyarrow_cython_example.pyx -> build/lib.linux-x86_64-3.7/pyarrow/tests
creating build/lib.linux-x86_64-3.7/pyarrow/tests/data
copying pyarrow/tests/data/v0.7.1.all-named-index.parquet -> build/lib.linux-x86_64-3.7/pyarrow/tests/data
copying pyarrow/tests/data/v0.7.1.column-metadata-handling.parquet -> build/lib.linux-x86_64-3.7/pyarrow/tests/data
copying pyarrow/tests/data/v0.7.1.parquet -> build/lib.linux-x86_64-3.7/pyarrow/tests/data
copying pyarrow/tests/data/v0.7.1.some-named-index.parquet -> build/lib.linux-x86_64-3.7/pyarrow/tests/data
running build_ext
creating build/temp.linux-x86_64-3.7
-- Runnning cmake for pyarrow
cmake -DPYTHON_EXECUTABLE=/root/Mehraj/cleaning/bin/python3.7  -DPYARROW_BOOST_USE_SHARED=on -DCMAKE_BUILD_TYPE=release /tmp/pip-install-hj1dj2ty/pyarrow
unable to execute 'cmake': No such file or directory
error: command 'cmake' failed with exit status 1

Failed building wheel for pyarrow
Running setup.py clean for pyarrow
Failed to build pyarrow
Installing collected packages: pymongo, ratelimit, pyparsing, kiwisolver, python-dateutil, cycler, matplotlib, py4j, pyspark, scipy, pytz, pandas, seaborn, backoff, humanize, future, tabulate, colorama, h2o-pysparkling-2.4, decorator, ipython-genutils, traitlets, parso, jedi, wcwidth, prompt-toolkit, backcall, pickleshare, ptyprocess, pexpect, pygments, ipython, nose, pika, findspark, MarkupSafe, Jinja2, click, itsdangerous, flask, pillow, multipledispatch, pyarrow, pyyaml, keras, vine, amqp, kombu, py, atomicwrites, attrs, pluggy, more-itertools, pytest, optimuspyspark
Running setup.py install for pyarrow ... error
Complete output from command /root/Mehraj/cleaning/bin/python3.7 -u -c "import setuptools, tokenize;file='/tmp/pip-install-hj1dj2ty/pyarrow/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-record-g9zs1p7w/install-record.txt --single-version-externally-managed --compile --install-headers /root/Mehraj/cleaning/include/site/python3.7/pyarrow:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/ipc.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/plasma.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/serialization.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/hdfs.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/types.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/parquet.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/util.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/filesystem.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/formatting.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/pandas_compat.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/orc.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/jvm.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/feather.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/compat.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/init.py -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/benchmark.py -> build/lib.linux-x86_64-3.7/pyarrow
creating build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_types.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_table.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_hdfs.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/conftest.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_io.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_builder.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_parquet.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_feather.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/util.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_misc.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_deprecations.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/pandas_examples.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_plasma_tf_op.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_serialization.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_tensor.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_array.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_scalars.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_convert_builtin.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_plasma.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_schema.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/deserialize_buffer.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_jvm.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_convert_pandas.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_ipc.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/test_cython.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
copying pyarrow/tests/init.py -> build/lib.linux-x86_64-3.7/pyarrow/tests
running egg_info
writing pyarrow.egg-info/PKG-INFO
writing dependency_links to pyarrow.egg-info/dependency_links.txt
writing entry points to pyarrow.egg-info/entry_points.txt
writing requirements to pyarrow.egg-info/requires.txt
writing top-level names to pyarrow.egg-info/top_level.txt
reading manifest file 'pyarrow.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '.so' found anywhere in distribution
warning: no previously-included files matching '.pyc' found anywhere in distribution
warning: no previously-included files matching '~' found anywhere in distribution
warning: no previously-included files matching '#' found anywhere in distribution
warning: no previously-included files matching '.git*' found anywhere in distribution
warning: no previously-included files matching '.DS_Store' found anywhere in distribution
writing manifest file 'pyarrow.egg-info/SOURCES.txt'
copying pyarrow/init.pxd -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/_orc.cpp -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/_orc.pxd -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/_orc.pyx -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/_parquet.cpp -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/_parquet.pxd -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/_parquet.pyx -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/_plasma.cpp -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/_plasma.pyx -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/array.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/benchmark.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/builder.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/error.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/feather.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/io-hdfs.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/io.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/ipc.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/lib.cpp -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/lib.pxd -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/lib.pyx -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/lib_api.h -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/memory.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/plasma_store -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/public-api.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/scalar.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/serialization.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/table.pxi -> build/lib.linux-x86_64-3.7/pyarrow
copying pyarrow/types.pxi -> build/lib.linux-x86_64-3.7/pyarrow
creating build/lib.linux-x86_64-3.7/pyarrow/include
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/allocator.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/api.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/array.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/buffer.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/builder.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/compare.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/memory_pool.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/pretty_print.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/record_batch.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/status.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/stl.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/table.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/table_builder.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/tensor.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/test-util.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/type.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/type_fwd.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/type_traits.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/visitor.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
copying pyarrow/include/arrow/visitor_inline.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/adapters
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/adapters/orc
copying pyarrow/include/arrow/adapters/orc/adapter.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/adapters/orc
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/adapters/tensorflow
copying pyarrow/include/arrow/adapters/tensorflow/convert.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/adapters/tensorflow
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/compute
copying pyarrow/include/arrow/compute/api.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/compute
copying pyarrow/include/arrow/compute/context.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/compute
copying pyarrow/include/arrow/compute/kernel.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/compute
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/compute/kernels
copying pyarrow/include/arrow/compute/kernels/cast.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/compute/kernels
copying pyarrow/include/arrow/compute/kernels/hash.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/compute/kernels
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/io
copying pyarrow/include/arrow/io/api.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/io
copying pyarrow/include/arrow/io/buffered.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/io
copying pyarrow/include/arrow/io/file.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/io
copying pyarrow/include/arrow/io/hdfs.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/io
copying pyarrow/include/arrow/io/interfaces.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/io
copying pyarrow/include/arrow/io/memory.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/io
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/ipc
copying pyarrow/include/arrow/ipc/api.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/ipc
copying pyarrow/include/arrow/ipc/dictionary.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/ipc
copying pyarrow/include/arrow/ipc/feather.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/ipc
copying pyarrow/include/arrow/ipc/json.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/ipc
copying pyarrow/include/arrow/ipc/message.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/ipc
copying pyarrow/include/arrow/ipc/reader.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/ipc
copying pyarrow/include/arrow/ipc/writer.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/ipc
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/api.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/arrow_to_pandas.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/arrow_to_python.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/benchmark.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/builtin_convert.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/common.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/config.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/decimal.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/helpers.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/init.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/io.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/iterators.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/numpy_convert.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/numpy_interop.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/numpy_to_arrow.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/platform.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/pyarrow.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/python_to_arrow.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
copying pyarrow/include/arrow/python/type_traits.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/python
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/bit-stream-utils.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/bit-util.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/bpacking.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/checked_cast.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/compiler-util.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/compression.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/compression_brotli.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/compression_lz4.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/compression_snappy.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/compression_zlib.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/compression_zstd.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/cpu-info.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/decimal.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/hash-util.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/hash.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/io-util.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/key_value_metadata.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/lazy.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/logging.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/macros.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/memory.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/optional.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/parallel.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/recursive_wrapper.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/rle-encoding.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/sse-util.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/stl.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/string.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/thread-pool.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/type_traits.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/variant.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/variant_cast.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/variant_io.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/variant_visitor.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
copying pyarrow/include/arrow/util/visibility.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util
creating build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util/variant
copying pyarrow/include/arrow/util/variant/optional.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util/variant
copying pyarrow/include/arrow/util/variant/recursive_wrapper.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util/variant
copying pyarrow/include/arrow/util/variant/variant_cast.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util/variant
copying pyarrow/include/arrow/util/variant/variant_io.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util/variant
copying pyarrow/include/arrow/util/variant/variant_visitor.h -> build/lib.linux-x86_64-3.7/pyarrow/include/arrow/util/variant
creating build/lib.linux-x86_64-3.7/pyarrow/includes
copying pyarrow/includes/init.pxd -> build/lib.linux-x86_64-3.7/pyarrow/includes
copying pyarrow/includes/common.pxd -> build/lib.linux-x86_64-3.7/pyarrow/includes
copying pyarrow/includes/libarrow.pxd -> build/lib.linux-x86_64-3.7/pyarrow/includes
creating build/lib.linux-x86_64-3.7/pyarrow/tensorflow
copying pyarrow/tensorflow/build.sh -> build/lib.linux-x86_64-3.7/pyarrow/tensorflow
copying pyarrow/tensorflow/plasma_op.cc -> build/lib.linux-x86_64-3.7/pyarrow/tensorflow
copying pyarrow/tests/pyarrow_cython_example.pyx -> build/lib.linux-x86_64-3.7/pyarrow/tests
creating build/lib.linux-x86_64-3.7/pyarrow/tests/data
copying pyarrow/tests/data/v0.7.1.all-named-index.parquet -> build/lib.linux-x86_64-3.7/pyarrow/tests/data
copying pyarrow/tests/data/v0.7.1.column-metadata-handling.parquet -> build/lib.linux-x86_64-3.7/pyarrow/tests/data
copying pyarrow/tests/data/v0.7.1.parquet -> build/lib.linux-x86_64-3.7/pyarrow/tests/data
copying pyarrow/tests/data/v0.7.1.some-named-index.parquet -> build/lib.linux-x86_64-3.7/pyarrow/tests/data
running build_ext
creating build/temp.linux-x86_64-3.7
-- Runnning cmake for pyarrow
cmake -DPYTHON_EXECUTABLE=/root/Mehraj/cleaning/bin/python3.7  -DPYARROW_BOOST_USE_SHARED=on -DCMAKE_BUILD_TYPE=release /tmp/pip-install-hj1dj2ty/pyarrow
unable to execute 'cmake': No such file or directory
error: command 'cmake' failed with exit status 1
----------------------------------------

Command "/root/Mehraj/cleaning/bin/python3.7 -u -c "import setuptools, tokenize;file='/tmp/pip-install-hj1dj2ty/pyarrow/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-record-g9zs1p7w/install-record.txt --single-version-externally-managed --compile --install-headers /root/Mehraj/cleaning/include/site/python3.7/pyarrow" failed with error code 1 in /tmp/pip-install-hj1dj2ty/pyarrow/

Hi @nadeemnehru,
It seems to be related with an error with pyarrow 0.10 and python 3.7
Is it possible for you to downgrade python to 3.6?

Hi @nadeemnehru,
It seems to be related with an error with pyarrow 0.10 and python 3.7
Is it possible for you to downgrade python to 3.6?

Thanks @argenisleon
I created a virtual environment with Python 3.6.8 and proceeded with installing Optimus. The installation completed.
Also when trying to start the Optimus, I get the following

from optimus import Optimus
JAVA_HOME is not set
Traceback (most recent call last):
File "", line 1, in
File "/root/Mehraj/cleaning368/lib/python3.6/site-packages/optimus/init.py", line 3, in
from .optimus import *
File "/root/Mehraj/cleaning368/lib/python3.6/site-packages/optimus/optimus.py", line 11, in
from optimus.ml.models import ML
File "/root/Mehraj/cleaning368/lib/python3.6/site-packages/optimus/ml/models.py", line 13, in
spark = SparkSession.builder.getOrCreate()
File "/root/Mehraj/cleaning368/lib/python3.6/site-packages/pyspark/sql/session.py", line 173, in getOrCreate
sc = SparkContext.getOrCreate(sparkConf)
File "/root/Mehraj/cleaning368/lib/python3.6/site-packages/pyspark/context.py", line 349, in getOrCreate
SparkContext(conf=conf or SparkConf())
File "/root/Mehraj/cleaning368/lib/python3.6/site-packages/pyspark/context.py", line 115, in init
SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)
File "/root/Mehraj/cleaning368/lib/python3.6/site-packages/pyspark/context.py", line 298, in _ensure_initialized
SparkContext._gateway = gateway or launch_gateway(conf)
File "/root/Mehraj/cleaning368/lib/python3.6/site-packages/pyspark/java_gateway.py", line 94, in launch_gateway
raise Exception("Java gateway process exited before sending its port number")
Exception: Java gateway process exited before sending its port number

Hi @nadeemnehru,
Are you using java 8?

@argenisleon
Think am using Java 9


Fix sample project

Coverage remained the same at 70.43% when pulling c86ff0e1825cee979c7532dd7137cf99b31440ab on sample-project into cfee8eba388e4d9c82800d14cb5b6d255d58186a on master.


temporary watermark

I need to add watermark temporarily while displaying images in django so that user can't do (save image as) and get the original image until purchased. I dont want to save image copy.

Okay I am continuing with OpenCV.


Beta into alpha

Description
release alpha with latest beta changes
Breaking Changes
JIRA Link
Checklist

[ ] Updated the Readme.md (if required) ?

:tada: This PR is included in version 22.0.0-alpha.1 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 22.0.0-beta.1 :tada:
The release is available on:

npm package (@beta dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


custom Field isEmpty check

added is Empty check for custom fields

Hey @salvarubiomartinez! Thank you!
It looks like you made a bunch of changes on top of the isEmpty check. Could you maybe split them in multiple PRs?

Hey @hecrj
Excuse me. I only want to merge the commit for checking isEmpty on custom fields.
I created another pull request with this commit.


SMD-665 - Integrate PF transformation into pipeline

Ticket
SMD-665 - Integrate PF transformation into pipeline
Summary of changes

The actual integration of transformation itself is just one line of code in ingest.py: data_to_load = pathfinders_transform(tables, reporting_round) - this is the only "functional" change to the code made in this PR
However adding this line causes a bunch of errors with existing tests, as changes have been made to the spreadsheet and schema while various components of the pipeline have been developed
So the vast majority of changes in this PR are around bringing the extraction and validation config dict up to date with the spreadsheet, and then propagating changes to column names or whatever throughout the rest of the pipeline

Apologies if this is done somewhere I can't see, but with regards to the column names, they should match what we find in the respective DataClass for each table in core/controllers/mappings.py. If they don't match these, then the pipeline will break with a KeyError.

Thanks @Joel-Pearce, could you clarify which column names in particular you are referring to?


Migrate implementations.md and how-to-help.md plus cleanup

Related Issue(s): #796 #767 #793
Proposed Changes:

migrated how-to-help to https://github.com/stac-utils/stac-ecosystem
removed implementations.md, migrating to stac-site https://github.com/radiantearth/stac-site/pull/31

PR Checklist:

[x] This PR is made against the dev branch (all proposed changes except releases should be against dev, not master).
[x] This PR has no breaking changes.
[x] I have added my changes to the CHANGELOG or a CHANGELOG entry is not required.
[ ] This PR affects the STAC API spec, and I have opened issue/PR #XXX to track the change.

Yeah, I've been going back and forth on if how to help should go on STAC site... I went with the repo because I'm interested in making issues to better track the ecosystem activity. Tracking the ecosystem (like software that implements stac) seems inappropriate for the core repo, and would be confusing I think.
But if I don't get to actually really organizing then just putting it on stac site could make sense. And yeah, a higher level roadmap there would be a good update.

Agreed, although you also have issues for stac-site which could be used for tracking. If that's the only reason for the individual repo, I think we could also go with the site.

The site repo would feel a bit weird to me, seems like that should be issues with the site.
I'm imagining issues like:

GDAL/OGR plug-in for STAC
STAC Validator updated to 1.0
Javascript library for reading / writing STAC
NAIP STAC catalog

Maybe just a document on that stuff is fine... And then that'd definitely be on stac-site.


Request for Training/Fine-Tuning Code Release

I was very excited to see the release of this model, as I have been interested in adding breath detection as part of preprocessing audio.
I am interested in fine-tuning the model to better suit my specific use case. However, I noticed that the repository does not currently include any code or documentation related to training or fine-tuning the model.
Could you please let me know if there are any plans to release the training or fine-tuning code? Having access to these resources would be immensely helpful for those of us looking to adapt the model to our unique requirements.
Thank you again for your hard work and contribution to the open-source community. I look forward to any updates you can provide.
Thanks!

Thank you for your kind words and support!
I'm currently quite busy with some upcoming deadlines, so I may not be able to provide further updates on training or fine-tuning in the repository until October. And our training methods involve self-training, which makes it challenging to organize the code concisely.
I was wondering whether it would be helpful to provide a simple fine-tuning code, even if it needs annotated breath data.
Thanks!

Any example fine-tuning code (even requiring labelled data) would be appreciated!

@ydqmkkx this project is great! Are you still planning on putting together some of the training code for this project?


Bug with flashing files

Issue with uploading files included in subfolders — doubled backslash with following 'b' character transforms into special symbol (in my case). And it freezes process of the uploading (my bad, it's just relatively big, and in this regard, I suggest a progress bar)
And why it's doubled in some cases and not in others?

Images

another example:

i've had the same error with MicroWebServer2 - upload of urlUtils.py crashed
All this is due to path replacement in microupload.py:
in make_dirs method path separator is replaced to '/' in line 96:
posix_path = path.replace(os.path.sep, '/')
but in main upload method it's not happening: lines 76 to 79:
if remote_dir:
      make_dirs(files, remote_dir, created_cache)
  with open(local_path, 'rb') as fd:
      files.put(remote_path, fd.read())

if you replace path separator in remote_path file upload will work correct:
files.put(remote_path.replace(os.path.sep, '/'), fd.read())

Having the same issue (ESP32).


Entry not found in cache

Repro steps:
Working with the extension during a day, the error is encountered frequently. It is hard to trigger a refresh. Seems like refresh is satisfied by the incomplete cache

Action: azureFunctions.deploy
Error type: Error
Error Message: Entry not found in cache.
Version: 0.17.0
OS: win32
Product: Visual Studio Code
Product Version: 1.33.1
Language: en

Call Stack
a.module.exports.a.createError extension.js:1:49013
b.<anonymous> extension.js:229:176642
extension.js:229:171257extension.js:229:171257
extension.js:229:43364extension.js:229:43364
extension.js:229:42231extension.js:229:42231
B.module.exports.e.exports.createRequestHandler [as _callback] extension.js:1:182121
B.module.exports.B.init.t._callback.t.callback.t.callback extension.js:181:5498
B.emit events.js:182:13
B.<anonymous> extension.js:181:18326
B.emit events.js:182:13
IncomingMessage.<anonymous> extension.js:181:17168
Object.onceWrapper events.js:273:13
IncomingMessage.emit events.js:187:15
endReadableNT _stream_readable.js:1090:12
process._tickCallback next_tick.js:63:19

Thanks for filing - as a workaround you could try signing out and signing back in to your Azure Account using the command palette.
Unfortunately there's not much we can do in this extension, but this is being tracked upstream here: https://github.com/microsoft/vscode-azure-account/issues/53


Mount error while using volume driver in Ubuntu 16.10

I am getting a mount error while using azurefile volume driver.  The system details are as follows
OS: Ubuntu 16.10
Docker version: 1.12.6
Azurefile version: 0.5.1
I installed the plugin as per instructions.  cifs-utils was installed as well.
The azurefile service as well as docker service is running.  I created a volume using
docker volume create -d azurefile --name test -o share=test
At this point I got the following error in the log for azurefile service
Apr 21 17:49:14 ubuntu azfile[28162]: time="2017-04-21T17:49:14+05:30" level=error msg="could not fetch metadata: cannot read metadata: open /etc/docker/plugins/azurefile/volumes/test1: no such file or director
y" name=test1 operation=get
Apr 21 17:49:14 ubuntu azfile[28162]: time="2017-04-21T17:49:14+05:30" level=debug msg="request accepted" name=test1 operation=create options=map[share:test1b]
Apr 21 17:49:14 ubuntu azfile[28162]: time="2017-04-21T17:49:14+05:30" level=info msg="created azure file share \"test1b\"" name=test1 operation=create options=map[share:test1b]
Apr 21 17:49:14 ubuntu azfile[28162]: time="2017-04-21T17:49:14+05:30" level=debug msg="request accepted" name=test1 operation=path

However the file share was created successfully in azure
Then I tried using the volume with
docker volume run -it --rm -v test:/data busybox
This resulted in a long hang by docker.  Meanwhile the following logs were generated
Apr 21 17:50:43 ubuntu azfile[28162]: time="2017-04-21T17:50:43+05:30" level=debug msg="request accepted" name=test1 operation=mount
Apr 21 17:52:45 ubuntu azfile[28162]: time="2017-04-21T17:52:45+05:30" level=error msg="mount failed: exit status 32\noutput=\"mount.cifs kernel mount options: ip=<IP_ADDRESS>,unc=\\\\\\\\glydata.file.core.wi
ndows.net\\\\test1b,vers=3.0,file_mode=0777,dir_mode=0777,uid=0,gid=0,user=glydata,pass=********\\nmount error(11): Resource temporarily unavailable\\nRefer to the mount.cifs(8) manual page (e.g. man mount.cifs
)\\n\"" name=test1 operation=mount
Apr 21 17:52:45 ubuntu dockerd[28173]: time="2017-04-21T17:52:45.801179559+05:30" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/a02a7b0ce64f9c238351cea1456c7d542ee9908b3a18bc8e
4aa12d7b88781abc: exec: \"auplink\": executable file not found in $PATH"
Apr 21 17:52:45 ubuntu azfile[28162]: time="2017-04-21T17:52:45+05:30" level=debug msg="request accepted" name=test1 operation=unmount
Apr 21 17:52:45 ubuntu azfile[28162]: time="2017-04-21T17:52:45+05:30" level=error msg="unmount failed: exit status 32\noutput=\"umount: /var/run/docker/volumedriver/azurefile/test1: not mounted\\n\"" name=test
1 operation=unmount
Apr 21 17:52:45 ubuntu dockerd[28173]: time="2017-04-21T17:52:45.829958160+05:30" level=warning msg="1b69f011fabb0f685fe5b38ea6c35376ffacf675f6578c7771f41f4668153026 cleanup: Failed to umount volumes: VolumeDri
ver.Unmount: unmount failed: exit status 32\noutput=\"umount: /var/run/docker/volumedriver/azurefile/test1: not mounted\\n\""
Apr 21 17:52:45 ubuntu dockerd[28173]: time="2017-04-21T17:52:45.830006784+05:30" level=error msg="Handler for POST /v1.24/containers/1b69f011fabb0f685fe5b38ea6c35376ffacf675f6578c7771f41f4668153026/start retur
ned error: VolumeDriver.Mount: mount failed: exit status 32\noutput=\"mount.cifs kernel mount options: ip=<IP_ADDRESS>,unc=\\\\\\\\glydata.file.core.windows.net\\\\test1b,vers=3.0,file_mode=0777,dir_mode=0777
,uid=0,gid=0,user=glydata,pass=********\\nmount error(11): Resource temporarily unavailable\\nRefer to the mount.cifs(8) manual page (e.g. man mount.cifs)\\n\""

The same set of operations runs successfully on Ubuntu 16.04 with the same docker and plugin versions

I have this issue in:

Ubuntu 16.04
Kernel 4.10
Docker 17.03.2-ee-5
Azurefile version: 0.5.1


[WIP] tests: added template + metadata functional test

Covers https://github.com/coreos/fleet/issues/1446

@jonboulle reverting of PR #1376 resolved the issue. We can keep these tests and merge this PR. Or I can spend more time and fix initial PR. What is your decision?

#1376 did not make it into a release right? If not, let's revert that + add the test for this release, and then we can rework a proper version for the next release. Thanks!

@jonboulle no, it didn't. tests cover the bug. new #1514 issue has been created. waiting for LGTM ;)

@jonboulle Done.

@jonboulle I discussed this with @kayrus, so maybe spend some time on it? cause it's really an optimization boost, previous logic triggered unnecessary RPC and I/O work. The code that can be fixed was located, and there is already a functional test for it, otherwise I'm ok.

@tixxdz I think your last comment should rather go into the newly opened issue, https://github.com/coreos/fleet/issues/1514 ? I was quite confused what it refers to at first...

I think you're right.
@kayrus thanks, this is great!

@kayrus some formal nitpicks: first of all, please update the title of this PR -- it doesn't apply anymore, and it's very confusing...
I think the new tests should be applied after the revert, to preserve bisectability.
Also, this might be a matter of taste -- but I believe it would be better to have only one revert commit covering both of the original commits (i.e. using revert -n); and with a proper commit message explaining why they are reverted...

Meh, too late ;-)

Sorry, all fair comments, just wanted to unblock.
On Wed, Mar 23, 2016 at 4:09 PM Olaf Buddenhagen<EMAIL_ADDRESS>wrote:

@kayrus https://github.com/kayrus some formal nitpicks: first of all,
please update the title of this PR -- it doesn't apply anymore, and it's
very confusing...
I think the new tests should be applied after the revert, to preserve
bisectability.
Also, this might be a matter of taste -- but I believe it would be better
to have only one revert commit covering both of the original commits (i.e.
using revert -n); and with a proper commit message explaining why they
are reverted...
—
You are receiving this because you modified the open/close state.
Reply to this email directly or view it on GitHub
https://github.com/coreos/fleet/pull/1512#issuecomment-200386656


Vue 2.0

I was wondering if Vue 2.0 conversion would be on the roadmap at some point?
Did I already ask this question? I just got a lot of deja vu and I went looking through the issues, but didn't find anything. :smile:
Scott

Vue 2.0 is definitely on the roadmap. Just not now. Waiting for the ecosystem to grow and become stable. I personally hate it when I stumble on packages that are not yet stable and find different bugs.

Cool. And for sure, you should wait until Vue 2.0 is clearly stable. But, growing the ecosystem might even depend on Vue 2.0 being used. :smile:
Scott

Reopened issue so other know this thing is scheduled.

FYI, The guys over at VueJS just released this migration guide.

Saw that. Will see what I can do about it. I hardly wait to port everything to Vue 2.0 myself :)

@rstoenescu Probably hard to answer, but could you give an estimate when the Quasar Framework with vue 2.0 version will be there? As it could make me head for vue 2.0 at this time.

I will probably start working on this one in one month top. Just to be sure, let's say it will be ready in 2 months. It all depends on how much money I can raise on Patreon so I can avoid working for others too to support day to day living :)

Looked into Vue 2.0 and used the migration helper to see what needs to be changed. Will start working on this with first chance I get.

Created vue2 branch to handle this.

Actively working on this on v0.8 branch now

Amazing work done. Moving to v0.8 right now ! :)

Thx! Great!

Yes. Appreciate hard and good work a lot. Vue is taking off like a rocket and that means frameworks like Quasar will be able ride the wave!
Scott

I sure hope so!


[question] Monitoring a custom environment

I am training an A2C algorithm on a custom environment using multiprocessing and SubprocVecEnv as follows:
`env = SubprocVecEnv([lambda: CustomEnv(args, i) for i in range(args.cpus)])
model = A2C(MlpLnLstmPolicy, env, verbose=1, tensorboard_log=None, learning_rate=7e-3, lr_schedule="linear")

model.learn(total_timesteps=args.training_steps, log_interval=10)

I want to monitor the learning and save model checkpoints using a Monitor and callbacks, however I can't seem to figure out how to combine everything.
I've tried doing
`env = SubprocVecEnv([lambda: CustomEnv(args, i) for i in range(args.cpus)])
env = Monitor(env, log_dir, allow_early_resets=True)
model = A2C(MlpLnLstmPolicy, env, verbose=1, tensorboard_log=None, learning_rate=7e-3, lr_schedule="linear")

model.learn(total_timesteps=args.training_steps, log_interval=10, callback=callback)`

but I get the following exception:

Traceback (most recent call last):
File "/main_file.py", line 96, in
env = train(args)
File "/main_file.py", line 76, in train
env = Monitor(env, log_dir, allow_early_resets=True)
File "/miniconda3/envs/RL/lib/python3.7/site-packages/stable_baselines/bench/monitor.py", line 27, in init
Wrapper.init(self, env=env)
File "/miniconda3/envs/RL/lib/python3.7/site-packages/gym/core.py", line 210, in init
self.reward_range = self.env.reward_range
AttributeError: 'SubprocVecEnv' object has no attribute 'reward_range'

So what is the correct way of using a monitor in this setting?

When following the example by applying the Monitor on the actual env and not the VecEnv it seems that the resulting log file gets corrupted. The callback fails with:
Traceback (most recent call last):   File "RoomControl_SubProcCallback.py", line 86, in <module>     model.learn(total_timesteps=total_timesteps, callback=callback)   File "/big/openai/stable-baselines/stable_baselines/ppo2/ppo2.py", line 400, in learn     if callback(locals(), globals()) is False:   File "RoomControl_SubProcCallback.py", line 48, in callback     x, y = ts2xy(load_results(log_dir), 'timesteps')   File "/big/openai/stable-baselines/stable_baselines/bench/monitor.py", line 180, in load_results     data_frame = pandas.read_csv(file_handler, index_col=None)   File "/big/innovation/venv/lib/python3.7/site-packages/pandas/io/parsers.py", line 685, in parser_f     return _read(filepath_or_buffer, kwds)   File "/big/innovation/venv/lib/python3.7/site-packages/pandas/io/parsers.py", line 463, in _read     data = parser.read(nrows)   File "/big/innovation/venv/lib/python3.7/site-packages/pandas/io/parsers.py", line 1154, in read     ret = self._engine.read(nrows)   File "/big/innovation/venv/lib/python3.7/site-packages/pandas/io/parsers.py", line 2059, in read     data = self._reader.read(nrows)   File "pandas/_libs/parsers.pyx", line 881, in pandas._libs.parsers.TextReader.read   File "pandas/_libs/parsers.pyx", line 896, in pandas._libs.parsers.TextReader._read_low_memory   File "pandas/_libs/parsers.pyx", line 950, in pandas._libs.parsers.TextReader._read_rows   File "pandas/_libs/parsers.pyx", line 937, in pandas._libs.parsers.TextReader._tokenize_rows   File "pandas/_libs/parsers.pyx", line 2132, in pandas._libs.parsers.raise_parser_error pandas.errors.ParserError: Error tokenizing data. C error: Expected 3 fields in line 3, saw 5
The contents of 'monitor.csv' is at the moment of failing:
#{"t_start":<PHONE_NUMBER>.5302866, "env_id": "RoomControl-v2"} 11398.178784,7200,121.623053131400.975874,7200,178.948676
What should the format of monitor.csv be?

@jbulow Please follow the rules and open if needed an issue with the issue template completely filled.
Note that it seems the error comes from your custom env. Please double check it. As mentioned in the readme, we don't do tech support nor consulting so your issue will be closed if it does not comes from stable baselines.

I could not find any documentation for Monitor other than the source. As you say, Monitor does not support multiple envs with common log file but does not fail when one tries to do that. I guess opening the log file with O_EXCL might fix the problem. Should I create an issue for this or does it work as intended? With the current design it's not straight-forward how to implement "save-new-best-model" in a vectorized setup.
Disclaimer: I'm very new to all this and thought that giving feedback was a way to improve stable baselines. In this context I don't have a clear understanding of what is considered "technical support". Point taken regarding opening issues using the template. (github suggested this defect when I was about to open a new issue)

I also ran into similar problems with monitoring vectorized environments. It was straight forward to tweak VecMonitor from OpenAI Baselines to work with Stable Baselines, as suggested above. This is what I ended up with.
from stable_baselines.common.vec_env import VecEnvWrapper
import numpy as np
import time
from collections import deque
import os.path as osp
import json
import csv

class VecMonitor(VecEnvWrapper):
    EXT = "monitor.csv"

    def __init__(self, venv, filename=None, keep_buf=0, info_keywords=()):
        VecEnvWrapper.__init__(self, venv)
        print('init vecmonitor: ',filename)
        self.eprets = None
        self.eplens = None
        self.epcount = 0
        self.tstart = time.time()
        if filename:
            self.results_writer = ResultsWriter(filename, header={'t_start': self.tstart},
                extra_keys=info_keywords)
        else:
            self.results_writer = None
        self.info_keywords = info_keywords
        self.keep_buf = keep_buf
        if self.keep_buf:
            self.epret_buf = deque([], maxlen=keep_buf)
            self.eplen_buf = deque([], maxlen=keep_buf)

    def reset(self):
        obs = self.venv.reset()
        self.eprets = np.zeros(self.num_envs, 'f')
        self.eplens = np.zeros(self.num_envs, 'i')
        return obs

    def step_wait(self):
        obs, rews, dones, infos = self.venv.step_wait()
        self.eprets += rews
        self.eplens += 1

        newinfos = list(infos[:])
        for i in range(len(dones)):
            if dones[i]:
                info = infos[i].copy()
                ret = self.eprets[i]
                eplen = self.eplens[i]
                epinfo = {'r': ret, 'l': eplen, 't': round(time.time() - self.tstart, 6)}
                for k in self.info_keywords:
                    epinfo[k] = info[k]
                info['episode'] = epinfo
                if self.keep_buf:
                    self.epret_buf.append(ret)
                    self.eplen_buf.append(eplen)
                self.epcount += 1
                self.eprets[i] = 0
                self.eplens[i] = 0
                if self.results_writer:
                    self.results_writer.write_row(epinfo)
                newinfos[i] = info
        return obs, rews, dones, newinfos

class ResultsWriter(object):
    def __init__(self, filename, header='', extra_keys=()):
        print('init resultswriter')
        self.extra_keys = extra_keys
        assert filename is not None
        if not filename.endswith(VecMonitor.EXT):
            if osp.isdir(filename):
                filename = osp.join(filename, VecMonitor.EXT)
            else:
                filename = filename #   + "." + VecMonitor.EXT
        self.f = open(filename, "wt")
        if isinstance(header, dict):
            header = '# {} \n'.format(json.dumps(header))
        self.f.write(header)
        self.logger = csv.DictWriter(self.f, fieldnames=('r', 'l', 't')+tuple(extra_keys))
        self.logger.writeheader()
        self.f.flush()

    def write_row(self, epinfo):
        if self.logger:
            self.logger.writerow(epinfo)
            self.f.flush()

@ajtanskanen Will you marry me?

btw, VecMonitoris now included in SB3: https://github.com/DLR-RM/stable-baselines3

Many thanks @ajtanskanen.

So in SB3 simply:
from stable_baselines3.common.vec_env import VecMonitor
env = VecMonitor(env, log_dir)

instead of
from stable_baselines3.common.monitor import Monitor
env = Monitor(env, log_dir)   # won't work with vectorized enviroments, will throw cryptic errors

sorry for off-topic, googled the error and got here

@araffin do you know which SB3 library version VecMonitor was introduced in? I may be using an older version because I still get monitoring errors.


Improve error messages for duplicate impl registration

If two types with the same typename implement traits with the same typename, it will be detected and an error will be printed to stderr. The error doesn't have enough information to debug the issue though, just that an issue occurred.
In debug, it should print information about where the error is coming from.

Fixed by 46b50c0b1eaaaef9d0a81da68785f0f67c7df23b


Add: option to install dcm2niix from source

Adds ability to install dcm2niix from source.
Example:
$ neurodocker generate -b debian:stretch -p apt --dcm2niix version=latest

Closes #105


ICA AllowMessages on Arabica is incorrect

Problem
On Arabica, the ICA host AllowMessages param is incorrect. It is currently a wildcard when it should be an allowlist of messages:
root@rootulp-arabica:~# celestia-appd query params subspace icahost AllowMessages
key: AllowMessages
subspace: icahost
value: '["*"]'

Wildcard is the default however it should've been overwritten with this list: https://github.com/celestiaorg/celestia-app/blob/2f74d9b84353b714295d3179b6df6c8315ffb813/app/ica_host.go#L3-L18

On Arabica and Mocha we can use governance to modify it but we should fix the override so that it doesn't need a governance vote on mainnet.

Please see meeting notes and hackMD doc for full details.
Summary:

Short term: we'll fix on Arabica, Mocha, and Mainnet via a governance proposal
Long term: in celestia-app v3.x we will hard-code the param to an explicit allowlist and make the param non governance modifiable
Note: this means there will be a ~1 week time period on mainnet where the ICA allowlist will be a wildcard instead of an explicit allowlist. We think this isn't a serious security concern b/c other Cosmos chains (Cosmos Hub, Neutron) have this param.

We're tackling this with three solutions:

https://github.com/celestiaorg/celestia-app/issues/3827
https://github.com/celestiaorg/celestia-app/issues/3836
https://github.com/celestiaorg/celestia-app/issues/3837


Add a Gitter chat badge to README.md

MickeyHub/ReplaceClassName now has a Chat Room on Gitter
@MickeyHub has just created a chat room. You can visit it here: https://gitter.im/MickeyHub/ReplaceClassName.
This pull-request adds this badge to your README.md:

If my aim is a little off, please let me know.
Happy chatting.
PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

Gitter suppot


Snyk issue found in request-promise-native dependency on<EMAIL_ADDRESS>

Hello @kr1sp1n
We're seeing Snyk reporting an issue with one of your dependencies (request-promise-native@1.0.7).  Looking at request-promise-native it looks like that library has been deprecated and will no longer be updated (a list of alternatives is provided on their github page).  Do you have a path to resolving this in the near future?  We'd like to continue using your library as our interface point for Vault, but need to have clean Snyk scans for our customers.
Looking forward to your reply.

Related to #150


Combobox support

Hello,
I have been looking at the wpf library for a bit now and I got to say it looks very nice. I don't have much experience with UI themes but even I can make my screen look more or less nice with it. I did note one UI components that seems to stick out and I was just wondering if you have any intent in the future to also provide a theme for Combobox.
Thank you for you time and keep up the good work, Chris

Yes, sure. I think a bootstrap style for WPF should be a good option for those who want a nice UI style but don't want to go too far, just like me. But too sad that the original author moved his interest away from WPF so I must be on my own and create this fork.
I'm maintaining this fork as a hobby project and also I use it myself in my paid job. I would like to add more controls, for example, ComboBox, but please give me some time.
At the same time, any help is welcomed.


Immutable options

https://github.com/Wiredcraft/loopback-connector-couchbaseX/blob/52d46f859474821318e327c4b926312d3eaf5332/lib/couchbaseX.js#L170
https://github.com/Wiredcraft/loopback-connector-couchbaseX/blob/52d46f859474821318e327c4b926312d3eaf5332/lib/couchbaseX.js#L177
I was stuck with a mystery bug about the options passed to view changed, and figure out it's was touched(some fields removed) by the code above.
Apparently the low-level library should not alter the parameter passed in, as the lib won't know how the param will be used.

@ChopperLee2011 could you make a fix?
cc @superbogy @makara

@xavierchow  I think the idea is: these options are handled by former call() or apply() method, so remove 'cause it is used and no need later.
I agree the lib should better not update these options, but could you describe your case with more details to let us know why that's wrong. thanks

https://github.com/Wiredcraft/omni-oms/pull/126/files#diff-3135543a523a8166b2e0e8ef1463b6f7R44-R76
@ChopperLee2011 ^ the criteria is used in two places, I have to revert the order of Object.assign to avoid it's overridden by the lib.

this is done, closing


Document permissions needed for this plugin

It would be nice if the permissions required for various operations are listed here. I'm using an API token with restricted permissions and currently it's not fully clear which permissions are necessary to add or not - it's a bit trial-and-error to figure it out.

This is still an issue.