Apple M1 Chip problem

[fatihdinc]

It appears that there is a problem with the current version of Tensorflow used by Cascade with the Apple M1 chips. There seems to be solutions to the problem, but this would require updating both the Tensorflow and the Python versions used for Cascade. I would really appreciate if you could consider addressing this issue in the future. Thanks!

[PTRRupprecht]

Hi @fatihdinc,

Thanks for pointing this out. I have myself no access to a Mac, but I will ask a colleague (who is on Holidays until next week) to evaluate the situation and maybe come up with a guideline or at least with an analysis of this situation within the next weeks. It seems that Tensorflow is currently not supporting the M1 chip right now officially, so one open question is also whether it makes sense to fix the problem. But we will come back to you with a better idea how to proceed.

Peter

[hluetck]

Hello @fatihdinc ,

we can give it a try to work out the configuration for Cascade on Macs with Apple Silicon, but it will likely take some time, due to limited hardware availability. In the meantime, here are 2 workarounds that you could try:

1. Docker Installation: Docker is now fully supported on Macs with Apple Silicon (see https://docs.docker.com/desktop/mac/apple-silicon/ ). Try to install Docker on your Mac and then follow the installation instructions as described in the Readme_Docker.md file in the etc folder of the repository (https://github.com/HelmchenLabSoftware/Cascade/blob/master/etc/README_Docker.md ).
2. Use the x86 emulation mode of your Mac (Rosetta 2). You can do this either by prefacing commands with arch -x86_64 or by creating a separate Rosetta 2 Terminal (see https://medium.com/swlh/run-x86-terminal-apps-like-homebrew-on-your-new-m1-mac-73bdc9b0f343 ). Once you have the Rosetta 2 terminal, you can install the x86 version of conda for macOS and then setup the environment as described in the Cascade installation instructions for macOS.

Please note that in both cases, the performance will likely be suboptimal and you will not use the full power of your ARM CPU.

Best, Henry

[MarziyehPourmousavi]

Hello @hluetck, I have a problem with checking packages in Jupyter-Notebook code. When I ran it, the kernel died. I installed tensorflow V2.1.0 and keras V2.3.1 and ruamel.yaml with pip on my MacBook (M1). Python version is 3.6.6 and Jupyter-Notebook version is 6.4.10. This is the list of pip I installed:

Package Version

---

absl-py 0.15.0
appnope 0.1.3
argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0
astor 0.8.1
astunparse 1.6.3
async-generator 1.10
attrs 22.1.0
backcall 0.2.0
bleach 4.1.0
cached-property 1.5.2
cachetools 4.2.4
certifi 2021.5.30
cffi 1.15.1
charset-normalizer 2.0.12
clang 5.0
cycler 0.11.0
dataclasses 0.8
decorator 5.1.1
defusedxml 0.7.1
entrypoints 0.4
flatbuffers 1.12
gast 0.2.2
google-auth 1.35.0
google-auth-oauthlib 0.4.6
google-pasta 0.2.0
grpcio 1.48.2
h5py 2.10.0
idna 3.4
importlib-metadata 4.8.3
ipykernel 5.5.6
ipython 7.16.3
ipython-genutils 0.2.0
ipywidgets 7.7.2
jedi 0.17.2
Jinja2 3.0.3
jsonschema 3.2.0
jupyter 1.0.0
jupyter-client 7.1.2
jupyter-console 6.4.3
jupyter-core 4.9.2
jupyterlab-pygments 0.1.2
jupyterlab-widgets 1.1.1
Keras 2.3.1
Keras-Applications 1.0.8
Keras-Preprocessing 1.1.2
kiwisolver 1.3.1
Markdown 3.3.7
MarkupSafe 2.0.1
matplotlib 3.3.4
mistune 0.8.4
nbclient 0.5.9
nbconvert 6.0.7
nbformat 5.1.3
nest-asyncio 1.5.6
notebook 6.4.10
numpy 1.18.5
oauthlib 3.2.2
opt-einsum 3.3.0
packaging 21.3
pandocfilters 1.5.0
parso 0.7.1
pexpect 4.8.0
pickleshare 0.7.5
Pillow 8.4.0
pip 21.2.2
prometheus-client 0.15.0
prompt-toolkit 3.0.31
protobuf 3.19.6
ptyprocess 0.7.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pycparser 2.21
Pygments 2.13.0
pyparsing 3.0.7
pyrsistent 0.18.0
python-dateutil 2.8.2
PyYAML 6.0
pyzmq 24.0.1
qtconsole 5.2.2
QtPy 2.0.1
requests 2.27.1
requests-oauthlib 1.3.1
rsa 4.9
ruamel.yaml 0.17.21
ruamel.yaml.clib 0.2.7
scipy 1.4.1
Send2Trash 1.8.0
setuptools 58.0.4
six 1.15.0
tensorboard 2.1.1
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.1
tensorflow 2.1.0
tensorflow-estimator 2.1.0
termcolor 1.1.0
terminado 0.12.1
testpath 0.6.0
tornado 6.1
traitlets 4.3.3
typing-extensions 3.7.4.3
urllib3 1.26.12
wcwidth 0.2.5
webencodings 0.5.1
Werkzeug 2.0.3
wheel 0.37.1
widgetsnbextension 3.6.1
wrapt 1.12.1
zipp 3.6.0

Also, I have tried to run Cascade in Docker but when I want to build the container, get this error:
=> ERROR [3/3] RUN conda env update -q -f /tmp/environment_mac.yml && 365.7s

[3/3] RUN conda env update -q -f /tmp/environment_mac.yml && pip install --quiet --no-cache-dir -r /tmp/requirements_mac.txt && conda clean -y --all && fix-permissions "/opt/conda" && fix-permissions "/home/jovyan":
#6 3.041 Collecting package metadata (repodata.json): ...working... done
#6 179.0 Solving environment: ...working... /bin/bash: line 1: 8 Killed conda env update -q -f /tmp/environment_mac.yml

---

executor failed running [/bin/bash -o pipefail -c conda env update -q -f /tmp/environment_mac.yml && pip install --quiet --no-cache-dir -r /tmp/requirements_mac.txt && conda clean -y --all && fix-permissions "${CONDA_DIR}" && fix-permissions "/home/${NB_USER}"]: exit code: 137

[hluetck]

Hi @MarziyehPourmousavi ,
I believe that the 2 issues that you report are unrelated. The environment configuration for Cascade on Apple Silicon is still pending, because of lack of access to hardware.
The Docker issue is likely because it ran out of memory while building the Docker image. I could reproduce this on my own machine. To solve this problem, I created a new Dockerfile and Conda environment file, which should be more efficient. Could you please test if it works for you? The relevant files are on a new branch: https://github.com/HelmchenLabSoftware/Cascade/tree/docker_issue/etc

[MarziyehPourmousavi]

@hluetck I said about checking the packages because I thought the problem was related to not installing them properly and I tried to fix it with another method which is Docker, but even with the file you uploaded, it was not fixed and I got this error:

=> ERROR [3/4] RUN conda env create -f /tmp/environment_mac.yml 20.2s

[3/4] RUN conda env create -f /tmp/environment_mac.yml:
#7 0.707 Collecting package metadata (repodata.json): ...working... done
#7 19.03 Solving environment: ...working... failed
#7 19.03
#7 19.03 ResolvePackageNotFound:
#7 19.03 - tensorflow=2.4
#7 19.03

---

executor failed running [/bin/bash -o pipefail -c conda env create -f /tmp/environment_mac.yml]: exit code: 1

[hluetck]

@MarziyehPourmousavi : so this means that the version of Tensorflow is not available for your architecture. I am afraid that without access to a Mac with Apple Silicon, I cannot properly solve this issue.

[PTRRupprecht]

There are now more comprehensive instructions on how to install Cascade on the Macbook with Apple Silicon Chip: https://github.com/HelmchenLabSoftware/Cascade/blob/master/etc/Instructions_Mac2024.md. Kudos to Master's student Xusheng Fang for figuring out this solution and for documenting it!

[matushalak]

For people who don't want to deal with downloading the intel version etc., working on a M3 Max here with latest software update (Sequoia 15.3.2), and all I had to do was:

• change the python version in the yaml to python = 3.8
• remove spyder from yaml
• add tensorflow to the yaml without specifying version (also installed appropriate version of keras automatically)
• in a couple of the scripts change the imports to from tensorflow import keras and from keras.ABC import XYZ instead of import tensorflow.keras and from tensorfrom.keras.ABC import XYZ

After doing this and setting up the conda venv from the yaml file everything runs fine. With pip install tensorflow-metal you even get GPU acceleration. At least to me, this seems like a simpler solution and it runs in the native macOS terminal using Apple Silicon.

[PTRRupprecht]

Thanks, @matushalak ! I've linked your installation recipe in the readme file on the GitHub landing page of CASCADE.