Where should we put the statement to initialize our PATH, in ~/.bash_profile or ~/.bashrc, and why?

TL;DR: put non-idempotent statements such as adding directories to the PATH in ~/.bash_profile.

Where to place statements such as export PATH=~cs3214/bin:$PATH is a recurring topic that is the subject of much confusion on the Internet.

Fundamentally, since environment variables are per-process properties that are inherited from a parent process such as a shell, they need to be initialized every time you log into a system and start new processes. The same goes for shell settings such as aliases, etc. To perform this initialization, the shell will read a set of startup files when it starts.

There is one problem, however: some of these statements are not idempotent, that is, executing them more than once changes their outcome. For instance, export PATH=~cs3214/bin:$PATH, if run twice, would result in a PATH that starts with: /home/courses/cs3214/bin:/home/courses/cs3214/bin:.... because the directory was prepended twice.

To deal with this situation, the shell adjusts which startup files it reads depending on when and how it is invoked. This process is described in Section 6.2 of the bash manual.

If a user first logs onto a machine, the first shell that is started is a so-called "login" shell. A "login" shell is not a different kind of shell, it is simply a certain way of starting bash that conveys to bash that it should act in "login" shell mode. Specifically, bash will do so when the first character in its av[0] is a hyphen -.

    PID TTY      STAT   TIME COMMAND
1337700 ?        S      0:00 sshd: gback@pts/2
1337701 pts/2    Ss     0:00  \_ -bash

(Side note: now you understand why the exec* family of system calls distinguishes between the name of an executable and separately allows the caller to specify av[0] - the two don't need to be identical!)

As a login shell, bash will read ~/.bash_profile (see bash man page for more details.) This is where you would put non-idempotent initialization such as appending to the PATH.

Once the user is logged on, they may start additional shells. Try typing bash, for instance. These shells (whose ancestor is a login shell) will inherit the PATH settings since they are part of the environment. However, they will not automatically have such settings as aliases which are needed by the user.

Such interactive, non-login shells read the file ~/.bashrc and interpret the settings in it. To make sure that the first interactive, login shell gain the same settings, the bash manual recommends "sourcing" ~/.bashrc from it. That is, your ~/.bash_profile file should contain

if [ -f ~/.bashrc ]; then . ~/.bashrc; fi

The source command in bash opens a script file and reads and interprets the commands in it in the context of the current shell (it does not start a new shell process because then the effect of these commands would be lost when that process exits.) An alias for the source command is, btw, a single dot .

The bash shell (and other shells) not only execute in interactive mode where the receive input from users, but they are also used as a command interpreter to interpret shell scripts. Shell scripts are text files containing shell commands. When a new shell is started to interpret a shell script, the shell is said to be non-interactive. Non-interactive shells do not read either ~/.bash_profile or ~/.bashrc on startup. Note that starting such a shell is different from using the source command. Shell scripts use a #!/bin/bash or #!/bin/sh shebang at the beginning of the file. /bin/sh is btw a symbolic link to /bin/bash on most modern Linux machines. In other words, running /bin/sh will run /bin/bash but tell it to operate in compatibility mode with the original Unix shell that was called /bin/sh, the so-called Bourne shell.

In addition to your personal startup files, there exist system-wide startup files which the shell also reads. These startup files may provide good default settings in case users don't create their own startup files (on some systems), and they may also help set up the environment so that users can conveniently use the software installed on the machine. See ls /etc/profile.d/* for examples.