So can't you just delete files willy-nilly, without telling Git? NO. If you delete a tracked file by hand, Git WILL notice, since it keeps track of every tracked file in the working directory. The key is to tell Git about the deletion so it becomes part of the history.
git rm <filename>
That deletes the file AND stages the deletion in one fell swoop.