The HTML header is the HTML above the shopping cart html.
The HTML footer is the HTML below the shopping cart html.
The best way to edit the look of the shopping cart is to use the Shop Template. The other links allow you to futher customize the HTML of Headers & Footers for the following pages:
Login
Billing Information / Address Page
Checkout Page
Offline Page
Gift Certificate HTML Emails
Confirmation Page HTML Fields
Request Shipping Quote HTML Fields
To edit HTML on category pages navigate to: Manage Categories > Edit Category HTML / Title
Text
You also have the ability to edit the text of different fields throughout the shopping cart. To edit the text, find the appropiate page level of the text, find the field, and then edit the text.