User-agent: * Disallow: /visit* Disallow: /v?* Disallow: /*? Disallow: /pingback* User-agent: dotbot Disallow: / User-agent: Twitterbot Disallow: /visit* Disallow: /v?* Disallow: /pingback*