pytutorial/workflow/python_pure

Python installation (Windows / Linux)

{:.no_toc}

* TOC {:toc}

Goal

Install your very own Python and keep everything under control.

Questions to David Rotermund

We need to download a suitable Python version from https://www.python.org. In the case we want to use PyTorch check here or TensorFlow check here we need to check if the Python version is okay.

Windows

I downloaded this file: https://www.python.org/ftp/python/3.11.7/python-3.11.9-amd64.exe

Select "Add python.exe to Path" and then press "Install Now".

We need to update pip first:

python.exe -m pip install --upgrade pip

ffmpeg

You might need to create mp4 files. E.g. for saving matplotlib animations

Thus you need ffmpeg binaries. You find them here: https://www.gyan.dev/ffmpeg/builds/

I went for https://www.gyan.dev/ffmpeg/builds/packages/ffmpeg-6.0-full_build.7z

There are differnt way to install the ffmpeg binaries (ffmpeg.exe, ffplay.exe, and ffprobe.exe). The easiest way is to put them here: %LOCALAPPDATA%\Programs\Python\Python311

Linux

Please check the dependencies beforehand!!!

For Fedora:

Honestly, I didn't get this depencency thing working. And I couldn't find a solution. However, I am installing a huge amount of packages anyway such that the required stuff is part of it. :-)

sudo dnf install dnf-plugins-core  # install this to use 'dnf builddep'
sudo dnf builddep python3

Python installation (after the dependencies are solved):

mkdir PySource
cd PySource
wget https://www.python.org/ftp/python/3.12.3/Python-3.12.3.tgz
tar -xvzf Python-3.12.3.tgz
cd Python-3.12.3
./configure --prefix=/home/[YOURUSERNAME]/P3.12 --enable-optimizations
make -j 10
make install
cd /home/[YOURUSERNAME]/P3.12/bin
./pip3 install --upgrade pip

Bash (.bash_profile)

alias pyhton3=/home/[YOURUSERNAME]/P3.12/bin/python3
alias pyhton=/home/[YOURUSERNAME]/P3.12/bin/python3
alias ipyhton=/home/[YOURUSERNAME]/P3.12/bin/ipython
alias ipyhton3=/home/[YOURUSERNAME]/P3.12/bin/ipython
alias python3=/home/[YOURUSERNAME]/P3.12/bin/python3
alias python=/home/[YOURUSERNAME]/P3.12/bin/python3
alias ipython=/home/[YOURUSERNAME]/P3.12/bin/ipython
alias ipython3=/home/[YOURUSERNAME]/P3.12/bin/ipython
alias pip3=/home/[YOURUSERNAME]/P3.12/bin/pip3
alias pip=/home/[YOURUSERNAME]/P3.12/bin/pip3

And add /home/[YOURUSERNAME]/P3.12/bin to the export PATH= command.

Numba

{: .topic-optional} This is an optional topic!

The question is: Is numba broken in the moment or not? First we install numpy and then numba. We will observe if numba tried to change the numpy version.

Windows

pip install numpy

pip install numba

If so then you have to decide if you want to remove numba and the downgraded numpy version before continuing. I would remove it!

pip uninstall numpy numba

Linux

cd /home/[YOURUSERNAME]/P3.11/bin
./pip install numpy

cd /home/[YOURUSERNAME]/P3.11/bin
./pip install numba

If so then you have to decide if you want to remove numba and the downgraded numpy version before continuing. I would remove it!

cd /home/[YOURUSERNAME]/P3.11/bin
./pip uninstall numpy numba

Package
numba	Numba is an open source JIT compiler that translates a subset of Python and NumPy code into fast machine code.

Basic Packages you want to install

Windows

pip3 install numpy scipy pandas flake8 pep8-naming black matplotlib seaborn ipython jupyterlab mypy dataclasses-json dataconf ipympl pywavelets scikit-image opencv-python scikit-learn tqdm argh sympy jsmin pybind11 pybind11-stubgen pigar asciichartpy natsort ipykernel 

Linux

cd /home/[YOURUSERNAME]/P3.11/bin
./pip3 install numpy scipy pandas flake8 pep8-naming black matplotlib seaborn ipython jupyterlab mypy dataclasses-json dataconf ipympl pywavelets scikit-image opencv-python scikit-learn tqdm argh sympy jsmin pybind11 pybind11-stubgen pigar asciichartpy natsort ipykernel 

Package
numpy	The fundamental package for scientific computing with Python
scipy	Fundamental algorithms for scientific computing in Python
pandas	pandas is a fast, powerful, flexible and easy to use open source data analysis and manipulation tool
flake8	Your Tool For Style Guide Enforcement
pep8-naming	Check your code against PEP 8 naming conventions
black	The uncompromising code formatter
matplotlib	Visualization with Python
seaborn	seaborn: statistical data visualization
ipython	IPython provides a rich architecture for interactive computing
jupyterlab	JupyterLab is the latest web-based interactive development environment for notebooks, code, and data.
mypy	Mypy is an optional static type checker for Python that aims to combine the benefits of dynamic (or "duck") typing and static typing.
dataclasses-json	Easily serialize Data Classes to and from JSON
dataconf	Simple dataclasses configuration management for Python with hocon/json/yaml/properties/env-vars/dict support
ipympl	Matplotlib Jupyter Integration
pywavelets	PyWavelets is open source wavelet transform software for Python.
scikit-learn	Simple and efficient tools for predictive data analysis
scikit-image	Image processing in Python
opencv-python	Open Source Computer Vision Library
tqdm	A Fast, Extensible Progress Bar for Python and CLI
argh	Building a command-line interface? Found yourself uttering “argh!” while struggling with the API of argparse? Don’t want to lose its power but don’t need the complexity?
sympy	SymPy is a Python library for symbolic mathematics.
jsmin	JavaScript minifier
pybind11	Seamless operability between C++11 and Python
pybind11-stubgen	For mypy
pigar	A tool to generate requirements.txt for Python project, and more than that.
asciichartpy	Nice-looking lightweight console ASCII line charts
natsort	Simple yet flexible natural sorting in Python.
ipykernel	IPython Kernel for Jupyter

PyTorch

Check here for a installation configurator.

Note: Even if you don't want to use TensorFlow, you want to install it too due to the tensorboard package. Tensorboard allows to track the progress during learning.

Windows

CPU only

pip3 install torch torchvision torchaudio torchtext 

NVidia GPU

pip3 install torch torchvision torchaudio torchtext  --index-url https://download.pytorch.org/whl/cu121

Linux

CPU only

cd /home/[YOURUSERNAME]/P3.11/bin
./pip3 install torch torchvision torchaudio torchtext --index-url https://download.pytorch.org/whl/cpu

NVidia GPU

cd /home/[YOURUSERNAME]/P3.11/bin
./pip3 install torch torchvision torchaudio torchtext

Package
torch	PyTorch is a Python package that provides two high-level features: a. Tensor computation (like NumPy) with strong GPU acceleration b. Deep neural networks built on a tape-based autograd system
torchvision	The torchvision package consists of popular datasets, model architectures, and common image transformations for computer vision.
torchaudio	The aim of torchaudio is to apply PyTorch to the audio domain.
torchtext	Models, data loaders and abstractions for language processing, powered by PyTorch

TensorFlow

Windows

CPU only

pip3 install tensorflow

NVidia GPU

pip3 install tensorflow[and-cuda]

Linux

CPU only

cd /home/[YOURUSERNAME]/P3.11/bin
./pip3 install tensorflow

NVidia GPU

cd /home/[YOURUSERNAME]/P3.11/bin
./pip3 install tensorflow[and-cuda]

Package
tensorflow	TensorFlow is an open source software library for high performance numerical computation. Its flexible architecture allows easy deployment of computation across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices.