I think you can basically put them wherever you want so long as you set the appropriate template path.
You may want them stored under the "main" MSM installation or you may prefer, as I do, to keep them within each site's directory.
As an example I have an MSM installation running 8 sites and the directories are set up like so (missing out some of the directories for the sake of brevity);
htdocs
'main' site
index.php
admin.php
eesystem
expressionengine
templates
default_site (templates for the main site)
images
etc
site2
index.php
admin.php
templates
images
site3
index.php
admin.php
templates
images
site4
index.php
admin.php
templates
images
and so on...