This is an update to my original post Dynamic Robots with ASP.NET 2.0, which was written for IIS6. IIS7 makes this process even easier! Use this approach to deliver different robots.txt files depending on whether you site is being crawled in http or https (or just to present your robots.txt file programatically).
Let’s take a look…
While at work yesterday, I saw a whole bunch of code like if (memberid > 0) parameters.Add(new SqlParameter(“@memberid”, memberid)); if (groupid > 0) parameters.Add(new SqlParameter(“@groupid”, groupid)); if (industryid > 0) parameters.Add(new SqlParameter(“@industryid”, industryid)); … and so on. It would be easy enough to squeeze it down to something like InsertIfGz(“@memberid”, memberid); InsertIfGz(“@groupid”, groupid); but it [...]
I just installed Ubuntu 9.04 (64-bit) on a new machine and am currently in the middle of compiling LinuxSampler, and for some reason had to make source changes to the latest stable to get it to compile. For starters, I had to run configure with –prefix=/usr. Don’t know whether that’s a 64-bit thing, a me-being-noob [...]
I needed to do this recently and I couldn’t find any instructions on the web. There were bits and pieces here and there, but nothing all in one place – and you need to know what you are looking for.
So here it is – step by step explanation on how to create a dynamic robots.txt file (or any text file for that matter) with IIS 6.0 and ASP.NET 2.0
