You will find below the instructions to set up you computer for Data Science 🔥
It comes mostly from Le Wagon Data Science bootcamp 🙏
Please read them carefully and execute all commands in the following order. If you get stuck, don't hesitate to ask for help 🙋♀️
Let's start 🚀
Have you signed up to GitHub? If not, do it right away.
👉 Upload a picture and put your name correctly on your GitHub account. Please do this now, before you continue with this guide.
Before we start, we need to check that the version of Windows installed on your computer is compatible with this setup instructions.
To be able to set up your computer, you need to have Windows 10 or Windows 11 installed.
To check your Windows version:
- Press
Windows+R - Type
winver - Press
Enter
✔️ If the first words of this window are Windows 10 or Windows 11 you're good to go 👍
❌ If not, you cannot proceed with this setup. You have to upgrade to Windows 10 first 👇
Upgrade to Windows 10
- Download Windows 10 from Microsoft
- Install it. It should take roughly an hour, but this depends on your computer.
- When the installation is over, execute the commands above ☝️ to check that you now have Windows 10.
ℹ️ Windows 11 upgrade is rolling now, which means it may or may not be available for your computer just yet.
Once you're sure that you're using Windows 10 or 11, you need to install all the latest updates.
Open Windows Update:
- Press
Windows+R - Type
ms-settings:windowsupdate - Press
Enter - Click on
Check updates
✔️ If you see a green check mark and the message "You're up to date", you're good to go 👍
❌ If you have an error message about Windows not being able to apply updates, please ask for help.
Activate Windows Update Service to fix Updates
Some antiviruses and pieces of software deactivate the Update service we need, resulting in the error you see. Let's fix that!
- Press
Windows+R - Type
services.msc - Press
Enter - Double Click
Windows Update Service - Set its
StartuptoAutomatic - Click on
Start - Click on
OkThen let's try updates again!
Some of the tools we need to install have been release with the 1903 version or above of Windows 10 so we need to make sure you have at least this one.
- Press
Windows+R - Type
winver - Press
Enter
Check the Version number:
✔️ If it says at least 1903, you are good to go 👍
❌ If it is below 1903, please ask for help.
We need to ensure that the Virtualization options are enabled in the BIOS of your computer.
For many computers, this is already the case. Let's check:
- Press
Windows+R - Type
taskmgr - Press
Enter - Click on the
Performancetab - Click on
CPU
✔️ If you see "Virtualization: Enabled", you're good to go 👍
❌ If the line is missing or if the virtualization is disabled, please ask for help before trying to activate the Virtualization
Activate Virtualization
We need to access the BIOS / UEFI of the computer to activate it.
- Press
Windows + R - Type
shutdown.exe /r /o /t 1 - Press
Enter - Wait for the computer to shutdown
- Click on
Troubleshoot - Click on
Advanced Options - Click on
UEFI Firmware Settings - Click on
Restart
You need to activate the virtualization option for your processor here:
- Most of the time, in the advanced settings, the CPU settings, or the Northbridge settings
- The option can be called differently according to your computer:
- Intel:
Intel VT-x,Intel Virtualization Technology,Virtualization Extensions,Vanderpool... - AMD:
SVM ModeorAMD-V
- Intel:
- Save the changes after activation and reboot the computer through the appropriate option
WSL is the development environment we are using to run Ubuntu. You can learn more about WSL here.
ℹ️ The following instructions depend on your version of Windows. Please execute only the instructions corresponding to your version 👇
If you are running Windows 11, we will install WSL 2 and Ubuntu in one command through the Windows Terminal.
Ctrl + Shift + Enter key stroke to execute Windows Terminal with administrator privileges instead of just clicking on Okor pressing Enter.
- Press
Windows+R - Type
wt - Press
Ctrl+Shift+Enter
A blue terminal window will appear:
- Copy the following command (
Ctrl+C) - Paste it into the terminal window (
Ctrl+Vor by right-clicking in the window) - Run it by pressing
Enter
wsl --install✔️ If the command ran without any error, please restart your computer and continue below 👍
❌ If you encounter an error message (or if you see some text in red in the window), please ask for help.
If you are running Windows 10, we will first install WSL 1 through the PowerShell Terminal.
Ctrl + Shift + Enter key stroke to execute Windows PowerShell with administrator privileges instead of just clicking on Okor pressing Enter.
- Press
Windows+R - Type
powershell - Press
Ctrl+Shift+Enter
A blue terminal window will appear:
- Copy the following commands one by one (
Ctrl+C) - Paste them into the PowerShell window (
Ctrl+Vor by right-clicking in the window) - Run them by pressing
Enter
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linuxdism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestartdism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart✔️ If all three commands ran without any error, please restart your computer and continue below 👍
❌ If you encounter an error message (or if you see some text in red in the window), please ask for help.
If you are running Windows 10, we will then upgrade WSL to version 2.
Once your computer has restarted, we need to download the WSL2 installer.
- Go to the download page
- Download "WSL2 Linux kernel update package"
- Open the file you've just downloaded
- Click
Next - Click
Finish
✔️ If didn't encounter any error message, you're good to go 👍
❌ If you encounter the error "This update only applies to machines with the Windows Subsystem for Linux", right click on the program and select uninstall; you shall be able to install it normally this time.
If you are running Windows 10, we will set WSL default version to 2.
Now that WSL 2 is installed, let's make it the default version:
- Press
Windows+R - Type
cmd - Press
Enter
In the window which appears, type:
wsl --set-default-version 2✔️ If you see "The operation completed successfully", you can close this terminal and continue below 👍
❌ If the message you get is about Virtualization, please ask for help.
Enable Virtual Machine Platform Windows feature
Follow the steps described here until you enable Virtual Machine Platform and Windows Subsystem for Linux
Enable Hyper-V Windows feature
Follow the steps described here until you enable the group Hyper-V
ℹ️ The following instructions depend on your version of Windows. Please execute only the instructions corresponding to your version 👇
If you are running Windows 11, after restarting you computer, you should see a terminal window saying WSL is resuming the Ubuntu installation process. When it's done, Ubuntu will be launched.
If you are running Windows 10, let's install Ubuntu throught the Microsoft Store:
- Click on
Start - Type
Microsoft Store - Click on
Microsoft Storein the list - Search for
Ubuntuin the search bar - Select version without any number, just plain "Ubuntu"
- Click on
Install
Uninstall wrong versions of Ubuntu
To uninstall a wrong version of Ubuntu, you just have to go to the Installed Program List of Windows 10:
- Press
Windows+R - Type
ms-settings:appsfeatures - Press
Enter
Find the software to uninstall and click on the uninstall button.
Once the installation is finished, the Install button becomes a Launch button: click on it.
At first launch, you will be asked some information:
- Choose a username:
- one word
- lowercase
- no special characters
- for example:
firstname
- Choose a password
- Confirm your password
ENTER.
You can close the Ubuntu window now that it is installed on your computer.
- Press
Windows+R - Type
cmd - Press
Enter
Type the following command:
wsl -l -v✔️ If the version of Ubuntu WSL is 2, you are good to go 👍
❌ If the version of Ubuntu WSL is 1, we will need to convert it to version 2.
Convert Ubuntu WSL V1 to V2
In the Command Prompt window, type:
wsl --set-version Ubuntu 2✔️ After a few seconds, you should get the following message: The conversion is complete.
❌ If it does not work, we need to be sure that Ubuntu files are not compressed.
Check for Uncompressed Files
- Press
Windows+R - Type
%localappdata%\Packages - Press
Enter - Open the folder named
CanonicalGroupLimited.UbuntuonWindows... - Right Click on the
LocalStatefolder - Click on
Properties - Click on
Advanced - Make sure that the option
Compress contentis not ticked, then click onOk.
Apply changes to this folder only and try to convert the Ubuntu WSL version again.
❌ If the conversion still does not work, please ask for help..
You can now close this terminal window.
Let's install Visual Studio Code text editor.
- Go to Visual Studio Code download page.
- Click on "Windows" button
- Open the file you have just downloaded.
- Install it with few options:
When the installation is finished, launch VS Code.
To make VS Code interact properly with Ubuntu, let's install the Remote - WSL VS Code extension.
Open your Ubuntu terminal.
Copy-paste the following commands in the terminal:
code --install-extension ms-vscode-remote.remote-wslThen open VS Code from your terminal:
code .✔️ If you see WSL: Ubuntu in a green box in the bottom left corner of the VS Code window, you're good to go 👍
❌ Otherwise, please ask for help.
ℹ️ The following instructions depend on your version of Windows.
If you are running Windows 11, the Windows Terminal is already installed and you can proceed to the next section 👇
If you are running Windows 10, let's install Windows Terminal, a real modern terminal:
- Click on
Start - Type
Microsoft Store - Click on
Microsoft Storein the list - Search for
Windows Terminalin the search bar - Select Windows Terminal"
- Click on
Install
Uninstall wrong version of Windows Terminal
To uninstall a wrong version of Windows Terminal, you just have to go to the Installed Program List of Windows 10:
- Press
Windows+R - Type
ms-settings:appsfeatures - Press
Enter
Find the software to uninstall and click on the uninstall button.
Once the installation is finished, the Install button becomes a Launch button: click on it.
Let's make Ubuntu the default terminal of your Windows Terminal application.
Press Ctrl + ,
It should open the terminal settings:
- Change the default profile to "Ubuntu"
- Click on "Save"
- Click on "Open JSON file"
We have circle in red the part you will change:
First, let's ask Ubuntu to start directly inside your Ubuntu Home Directory instead of the Windows one:
- Locate the
"name": "Ubuntu", - Add the following line after it:
"commandline": "wsl.exe ~",Then, let's disable warning for copy-pasting commands between Windows and Ubuntu:
- Locate the line
"defaultProfile": "{2c4de342-...}" - Add the following line after it:
"multiLinePasteWarning": false,You can save these changes by pressing Ctrl + S
✔️ Your Windows Terminal is now setup 👍
This terminal has tabs: you can choose to open a new terminal tab by clicking on the + next to the current one.
From now on, every time we will refer to the terminal or the console it will be this one.
Let's install some useful extensions to VS Code.
code --install-extension ms-vscode.sublime-keybindings
code --install-extension emmanuelbeziat.vscode-great-icons
code --install-extension ms-python.python
code --install-extension KevinRose.vsc-python-indent
code --install-extension ms-python.vscode-pylance
code --install-extension ms-toolsai.jupyterHere is a list of the extensions you are installing:
git is a command line software used for version control.
To install git:
- Open a terminal
- Copy and paste the following commands:
sudo apt update
sudo apt install -y gitThese commands will ask for your password: type it in.
ENTER.
Let's now install GitHub official CLI (Command Line Interface). It's a software used to interact with your GitHub account via the command line.
In your terminal, copy-paste the following commands and type in your password if asked:
curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null
sudo apt update
sudo apt install -y ghTo check that gh has been successfully installed on your machine, you can run:
gh --version✔️ If you see gh version X.Y.Z (YYYY-MM-DD), you're good to go 👍
❌ Otherwise, please ask for help.
Instead of using the default bash shell, we will use zsh.
In a terminal execute the following command and type in your password if asked:
sudo apt install -y zsh curl vim imagemagick jq unzip treeLet's install the zsh plugin Oh My Zsh.
In a terminal execute the following command:
sh -c "$(curl -fsSL https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"If asked "Do you want to change your default shell to zsh?", press Y
At the end your terminal should look like this:
✔️ If it does, you can continue 👍
❌ Otherwise, please ask for help.
In a terminal execute the following command:
export CURRENT_DIR=`pwd`
export ZSH_PLUGINS_DIR="$HOME/.oh-my-zsh/custom/plugins"
mkdir -p "$ZSH_PLUGINS_DIR" && cd "$ZSH_PLUGINS_DIR"
if [ ! -d "$ZSH_PLUGINS_DIR/zsh-syntax-highlighting" ]; then
git clone https://github.com/zsh-users/zsh-autosuggestions
git clone https://github.com/zsh-users/zsh-syntax-highlighting
fi
cd "$CURRENT_DIR"CLI is the acronym of Command-line Interface.
In this section, we will use GitHub CLI to interact with GitHub directly from the terminal.
It should already be installed on your computer from the previous commands.
First in order to login, copy-paste the following command in your terminal:
email
gh auth login -s 'user:email' -wYou will get the following output:
! First copy your one-time code: 0EF9-D015
- Press Enter to open github.com in your browser...Select and copy the code (0EF9-D015 in the example), then press ENTER.
Your browser will open and ask you to authorize GitHub CLI to use your GitHub account. Accept and wait a bit.
Come back to the terminal, press ENTER again, and that's it.
To check that you are properly connected, type:
gh auth status✔️ If you get Logged in to github.com as <YOUR USERNAME> , then all good 👍
❌ If not, ask for help..
Then run the following configuration line:
gh config set git_protocol sshWe need to generate SSH keys which are going to be used by GitHub to authenticate you. You can think of it as a way to log in, but different from the well known username/password pair.
Open a terminal and copy-paste this command, replacing the email with yours (the same one you used to create your GitHub account).
mkdir -p ~/.ssh && ssh-keygen -t ed25519 -o -a 100 -f ~/.ssh/id_ed25519 -C "firstName.lastName@solution-bi.com"It will prompt for information. Just press enter until it asks for a passphrase.
ENTER.
Now, you will give your public key to GitHub.
In your terminal copy-paste the following command:
gh auth refresh -s write:public_keyIt will prompt a one time code (####-####) on the screen. Copy it and press ENTER, then paste the code in your browser and follow the instructions to Authorize GitHub.
Back in the terminal, press ENTER and run this:
gh ssh-key add ~/.ssh/id_ed25519.pubThis should return ✓ Public key added to your account. If not, do not hesitate to ask for help..
Programmers love to refine and polish their shell and tools. We'll start with a great default configuration.
-
Copy the content of the
gitconfig,aliases,profile,zprofile,zshrcfiles into~/.gitconfig,~/.aliases,~/.profile,~/.zprofile,~/.zshrc. -
If some of these files do not exist on your computer, just create them. TIPS: You can easily modify or create a file by running the following command, for e.g.
code ~/.aliases -
Copy the content of the
keybindings.jsonandsettings.jsonfiles into~/.vscode-server/data/Machine/keybindings.jsonand~/.vscode-server/data/Machine/settings.json
Then, open your terminal and run the following command:
export GITHUB_USERNAME=`gh api user | jq -r '.login'`
echo $GITHUB_USERNAMEYou should see your GitHub username printed. If it's not the case, stop here and ask for help.
There seems to be a problem with the previous step (gh auth).
Check the emails registered with your GitHub Account. You'll need to pick one at the next step:
gh api user/emails | jq -r '.[].email'Copy-paste the following command, replacing the first name, last name and email with yours.
☝️ Be careful you need to put one of the email listed above thanks to the previous gh api ... command.
git config --global user.email "firstName.lastName@solution-bi.com"
git config --global user.name "FirstName LastName"Please now quit all your opened terminal windows.
Installing Python (with pyenv)
Ubuntu comes with an outdated version of Python that we don't want to use. You might already have installed Anaconda or something else to tinker with Python and Data Science packages. All of this does not really matter as we are going to do a professional setup of Python where you'll be able to switch which version you want to use whenever you type python in the terminal.
First let's install pyenv with the following Terminal command:
git clone https://github.com/pyenv/pyenv.git ~/.pyenv
exec zshIgnore the pyenv: no such command 'virtualenv-init' for now.
Let's install some dependencies needed to build Python from pyenv:
sudo apt-get update; sudo apt-get install make build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm \
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev \
python-dev python3-devLet's install the latest stable version of Python supported by this setup:
pyenv install 3.8.12This command might take a while, this is perfectly normal. Go play a game of darts while you wait 🎯
OK once this command is complete, we are going to tell the system to use this version of Python by default. This is done with:
pyenv global 3.8.12
exec zshTo check if this worked, run python --version. If you see 3.8.12, perfect! If not, ask for help.
Before we start installing relevant Python packages, we will isolate the data science setup into a dedicated virtual environment. We will use a pyenv plugin called pyenv-virtualenv.
First let's install this plugin:
git clone https://github.com/pyenv/pyenv-virtualenv.git $(pyenv root)/plugins/pyenv-virtualenv
exec zshLet's create the virtual environment we are going to use during the whole bootcamp:
pyenv virtualenv 3.8.12 dataLet's now set the virtual environment with:
pyenv global dataGreat! Anytime we'll install Python package, we'll do it in that environment.
Now that we have a pristine data virtual environment, it's time to install some packages in it.
First, let's upgrade pip, the tool to install Python Packages from pypi.org. In the latest terminal where the virtualenv data is activated, run:
pip install --upgrade pipThen let's install some packages for the first weeks of the program:
pip install -r https://raw.githubusercontent.com/lewagon/data-setup/master/specs/releases/linux.txtLet's generate the configuration file for Jupyter Notebook...
jupyter notebook --generate-configWe will now edit the generated Jupyter configuration file:
code $HOME/.jupyter/jupyter_notebook_config.pyLocate the following line in the configuration file:
# c.NotebookApp.use_redirect_file = TrueAnd replace it with this one:
c.NotebookApp.use_redirect_file = FalseLet's try to run Jupyter:
jupyter notebookThis command should have opened a Jupyter page in your browser. If it is not the case, please ask for help.
To stop the Jupyter server in the terminal, press Ctrl + C, enter y, then press Enter.
Pimp your jupyter notebooks with awesome extensions:
# install nbextensions
jupyter contrib nbextension install --user
jupyter nbextension enable toc2/main
jupyter nbextension enable collapsible_headings/main
jupyter nbextension enable spellchecker/main
jupyter nbextension enable code_prettify/code_prettifyImprove the display of the details disclosure elements in your notebooks.
Open custom/custom.css in the config directory:
cd $(jupyter --config-dir)
mkdir -p custom
touch custom/custom.css
code custom/custom.cssEdit custom.css with:
summary {
cursor: pointer;
display:list-item;
}
summary::marker {
font-size: 1em;
}You can close VS Code.
Let's reset your terminal:
exec zshNow, check you can launch a notebook server on your machine:
jupyter notebookYour web browser should open on a jupyter window. Click on New. A tab should open on a new notebook.
Perform a sanity check for jupyter notebooks nbextensions. Click on Nbextensions:
Untick "disable configuration for nbextensions without explicit compatibility" then check that at least all nbextensions circled in red are enabled:
You can close your web browser then terminate the jupyter server with CTRL + C.
Check your Python version with the following commands:
zsh -c "$(curl -fsSL https://raw.githubusercontent.com/lewagon/data-setup/master/checks/python_checker.sh)" 3.8.12Run the following command to check if you successfully installed the required packages:
zsh -c "$(curl -fsSL https://raw.githubusercontent.com/lewagon/data-setup/master/checks/pip_check.sh)"Now run the following command to check if you can load these packages:
python -c "$(curl -fsSL https://raw.githubusercontent.com/lewagon/data-setup/master/checks/pip_check.py)"Make sure you can run Jupyter:
jupyter notebookAnd open a Python 3 notebook.
Make sure that you are running the correct python version in the notebook. Open a cell and run :
import sys; sys.versionHere you have it! A complete python virtual env with all the third-party packages you'll need for the whole bootcamp.
We need an easy way to transfer files from Windows to Ubuntu and vice versa.
In order to do that, let's create shortcuts to Ubuntu directories in the Windows File Explorer:
- Open the Windows File Explorer (or use the shortcut
WIN+E) - In the Address Bar, enter
\\wsl$\(or\\wsl$\Ubuntuif it does not work) - You now have acces to the Ubuntu file system
- Dive into the Ubuntu file system in order to look for directories of interest
- Drag the desired folders into the Address Bar in order to create shortcuts
Another option to move files around is to open the Windows File Explorer from the Ubuntu terminal:
- Open an Ubuntu terminal
- Go to the directory you wish to explore
- Run the
wslview . - If you get an input output error message, run
wsl --shutdownin a Windows PowerShell and reopen an Ubuntu terminal
You might want to figure out the exact location of a Windows directory in the Ubuntu file system, or the other way around.
In order to convert a Windows path to and from an Ubuntu path:
- Open an Ubuntu terminal
- Use the
wslpath "C:\Program Files"command in order to translate a Windows path into an Ubuntu path - Use the
wslpath -w "/home"command in order to translate an Ubuntu path into a Windows path - In particular, the
wslpath -w $(pwd)command returns the Windows path of the current Ubuntu directory
Some Python packages require a compiler to function properly. Let's install one:
If you're unsure about which system you're using please ask for help.