Can I put all my fields into one field group and show just the fields I need on each channel's publish view? Any future issues I might run into working this way?
Apart from performance there's also another problem:
Suppose your 1 "master field group" is being used by 4-5 channels, and you want to add one field to one of your channels, you have to add it to your master field group, and hide it in all the publish layouts of all your channels.
So if you use this master field group approach, you really need to have planned out your entire website structure before setting up your fields, because having to hide each field you add in 20 different channels becomes very dull, very quickly.
If you are doing content planning then I see no problems with this approach. For example, I use separate channels for each section of a web site e.g. 'About Us', 'Why use our services'. I use the same channel field group for each of these sections since they are all structured in the same way e.g. abstract, detail, useful links etc.
I use common sense to break away from using a master field group approach - for example, if my landing and internal pages for a given section of the web site have several fields that are different then I break away from the master field group approach more for the clients' experience - never good to have a bunch of fields that don't need input for some pages but do for others!
In essence, it's a common sense approach borne out from content planning.
I am a huge advocate of having a "master" field set.
You can use Publish Layout to show/hide fields, and use add-ons like Entry Layout to further customize & present separate layouts per entry within a channel.
Only have a separate field set for add-ons that truly require them, like channel-based member fields, Solspace Calendar, etc.
The performance hit on the entry form is more than compensated by truly modular templates that share same field names across channels.