# Mark's notes:
# /robots.txt file for http://metergallery.com
# 
# the user-agent field indicates which robots are covered, in this case all of them (*)
# the disallow field is a relative path to what you don't want 'roboted.'
# to add more disallowed items, create more Disallow: lines, don't just add more items to
# the current one.
User-agent: *
Disallow: /admin/
# NOTES (from webcrawler):
# The "/robots.txt" file usually contains a record looking like this: 
# 
#        User-agent: *
#        Disallow: /cgi-bin/
#        Disallow: /tmp/
#        Disallow: /~joe/
# 
# In this example, three directories are excluded. 
# 
# Note that you need a separate "Disallow" line for every URL
# prefix you want to exclude -- you cannot say "Disallow: /cgi-bin/ /tmp/".
# Also, you may not have blank lines in a record, as they are used to delimit
# multiple records. 
# 
# Note also that regular expression are not supported in either the User-agent
# or Disallow lines. The '*' in the User-agent field is a special value meaning "any robot".
# Specifically, you cannot have lines like "Disallow: /tmp/*" or "Disallow: *.gif".