I'd like to provide my users a prettier UI to publish and edit/delete content (compared to what is provided by EE's control panel). For example, say I wanted to divide my entry form into tabs, etc. What are my options?
You can build a custom "My Account" area in your EE templates to handle all editing of content. The primary addon you'd use is the Safecracker Module.
Here are some global variable conditionals you'll use a lot with a custom "My Account" area:
There is also alternative syntax for some of the above if you are trying to use the conditional inside a channel:entries tag... variable names get prefixed with "logged_in_"
You already can use tabs in your publish layout:
Edit an entry > click "show toolbar" > add/remove tabs in this section:
More info in the EE documentation: Publish Page Layouts