Tim Mackey's Weblog

IIS went down this morning after KB 958644 was isntalled automatically last night. My sites were showing "Service Unavailable"
there were several system event logs like so:

• A process serving application pool '.Net 2.0 App Pool' suffered a fatal communication error with the World Wide Web Publishing Service. The process id was '2960'. The data field contains the error number.
• Application pool '.Net 2.0 App Pool' is being automatically disabled due to a series of failures in the process(es) serving that application pool.
  

rebooting the server did not solve the problem.  so, on further research, KB article 885654 revealed the cause of the errors, lack of registry permissions for NETWORK SERVICE.  i hadn't changed any configuration on the server so assumed it was caused by a change in that windows update package.  i dug out ProcessMonitor and set up a registry filter for w3wp.exe where the Result was ACCESS DENIED.  there were about 50 entries during the time i enabled event capture and loaded up one of the sites that uses the affected application pool.  Most of them pointed to SystemCertificates entries.  I went through each one in regedit and gave NETWORK SERVICE full control where the process attempted to Create a key, and Read only access where the process attemped to Open the key.  worked fine then.

hope this helps someone out.

Windows has a pointless 32Gb sizelimitation on FAT32 partitions.
a very sound chap called Tom Thornill wrote a partition formatting utility that does not have this limit. full details on his site

if, like me, you were running VS 2008 Beta 2 on a server and you had crystal reports working fine, and then recompiled your projects in VS 2008 RTM, you may get an error like this when you try and create a crystal report on a production server:

System.Runtime.InteropServices.COMException (0x800736B1):
Retrieving the COM class factory for component with CLSID
{5FF57840-5172-4482-9CA3-541C7878AE0F} failed due to the following error:
800736b1. at CrystalDecisions.CrystalReports.Engine.ReportDocument..cctor()

In my case, i had installed Crystal Reports Runtime for VS 2008 Beta 2, and i never installed the runtime for the RTM version.  it should be available on your VS machine, if you opted for CR in the VS installer:
C:\Program Files\Microsoft SDKs\Windows\v6.0A\Bootstrapper\Packages\CrystalReports10_5

should probably uninstall the old version of the CR runtime while you're at it too. 

i started a thread on the DELL community forums asking about migrating my windows 2003 software RAID setup to a hardware RAID controller, without reinstalling windows.  you can read the thread to get the context for these documented instructions below:

success. after a marathon 7 hour session in the datacenter it finally works. however the whole thing was much too stressful for my liking!

my problems were significantly compounded by the lack of a CD drive, so i had to get my imaging software (Terabyte Unlimited 'Image') to boot via a USB drive, which was never going to be easy with a Dell BIOS etc. although to be fair it does work, once you understand that you must reboot after configuring the BIOS to recognise the USB drive, and then go into the bios again and tell it what order to boot in.

i thought it useful to document the steps i took in case anyone else is in a similar situation.

in summary, i wanted to do an in-place migration from software raid in windows 2003 to hardware raid with the dell/adaptec CERC 6 channel SATA RAID card. all without using a CD drive (trust me, get one if you ever have to do this. i tried but couldn't get the BIOS to recognise the drive).

i got a very helpful email from Terabyte support before starting the migration:
****************************************
Hello,
If you just have a single simple volume (once the software RAID is broke), you can remove the dynamic drive by changing the file system ID back to 7 (via BootItNG partition work/properties). If you backup while dynamic, it will just backup the whole dynamic partition sector by sector.
So what I'd personally do is:
1. break the raid, (maybe remove the secondary drive as a backup)
2. ensure it's a single simple volume that's left (I'm assuming you just have c,
3. boot BootItNG, change the file system ID to 7,
4. boot back to windows (if it didn't boot, just put back the file system id to dynamic),
5. install the CERC drivers (if needed),
6. do the image, with byte-for-byte validation (slower in windows because it thrashes a bit),
7. ensure I can access the image to restore with Image For Dos or Image For Linux.
8. Configure the hardware RAID,
9. restore the image (use additional option of TA when restoring the boot partition).
*****************************************

windows software RAID requires dynamic disks, which are not officially reversible back to basic disks. this further complicated the process because i wanted to go back to basic disks for the RAID. you can convert back to a basic disk as described above with BootItNG, and also via the DskProbe windows support tool, instructions available from: http://www.nthelp.com/NT6/dynamic.htm. although i also read that if you use software raid, you shouldn't use this approach, so i unplugged the second drive, booted into windows and removed the mirror, and the offline disk from Windows Disk Manager. Then i changed to a basic disk via DskProbe and it did work, although DiskManager reported a single drive, which was marked offline or something very worrying like that. however the files were all there and working etc.

i followed the instructions above, although i did install the RAID card before changing any drivers or removing the software RAID, and i got windows to load up the drivers for it, so it would work automatically once booting from the RAID controller. this was one less problem in the equation and i was keen to get it out of the way as early in the process as possible.

to get Image For Dos to boot off USB, i used the bundled 'makedisk' utility without any init strings. I configured the BIOS to emulate a USB drive as a hard drive, floppy did not work. Then rebooted and changed the boot sequence so that the USB drive came first.
the disk image (of the 'active' drive, the other was still unplugged) took about 30 minutes with byte-for-byte validation, ~25Gb of data, copying to an external USB drive, using ImageForDos. i had to change the keyboard to a PS2 connection because in my first attempt the program locked out the keyboard after identifying the USB drive.

then i verified that i could access the image with the restore option. and went ahead with setting up a RAID mirror in the CERC configuration utility. plugged in both drives into ports 4 and 5. incidentally i needed a longer SATA cable for drive0 because it only had a 4inch cable and the RAID plugs on the card are miles away from the drive bays. The cable for the other drive fit with a stretch into the last port on the card.
so i did a reboot, pressed Ctrl-A etc. initialise the 2 drives. set up a RAID 1 mirror as per the manual instructions. i used the quick-init option because i was really in a hurry to get the job done. i set high priority also. i did not 'clear' the drives, but this may have been no harm to do.

then i rebooted and loaded my restore image with ImageForDos. the 'TA' option referred to in the instructions marks the partition active and inserts a standard MBR. it took about 30 minutes again and finished successfully. i was a bit worried because the restore program asked me which drive did i want to restore to, i assumed it didn't matter and chose the second drive. my cause for concern was that i wasn't sure if the controller would do it's mirroring thing during this restore operation. my fears were doubled when i rebooted after the restore, and got a message something like "no operating system, F1 to continue, F2 to setup". at this stage i had about 5 minutes before 'visiting hours' were over in the datacentre and i would have had to take the server home for the weekend to get it ready before monday 9am, no thanks! i thought it had failed. i was about to pack up and go home when i went in to look in the bios one last time in case i missed anything. i didn't change anything, and i also inspected the RAID config via the BIOS utility, but again, didn't change anything. i tried to turn off caching as a desperate attempt to fix the boot problems, but it wouldn't accept the change for some reason. anyhow, i rebooted again, said a quiet prayer, and i couldn't believe it but windows started loading, i was never so happy to see that Windows logo appear on screen (you probably had to be there to understand, 7 hours of white noise from being surrounded by a gzillian racks, no lunch, very frustrated etc!)

now all is well with the 2 drives showing up 'optimal' in the DELL/Adaptec storage manager. i gather this means that the build/verify has already completed in the background. the URL for the storage manager is http://tinyurl.com/2nddta

i didn't want to install all that dell 'management' software that came with the RAID card.
another good thing i forgot to mention. windows disk manager now shows a healthy primary partition for C:. it doesn't say 'fault tolerant' anymore obviously because the software raid is gone and the hardware RAID is abstracted away from the windows disk manager.

hope this helps someone someday.

i found this insanely difficult because none of the supposedly normal options worked.  merge modules didn't seem suitable because i just wanted to install the CR dlls once and for all on the server.  according to an MSDN2 article i should be able to perform a windows installer deployment, but of course when i copy the key code from the VS2005 > Help > About dialog, and use it in the installer on the server, the error message is that the key code has expired or is invalid.  'business objects' appear to be forcing users into an upgrade path for their new product, by claiming that a 'compatibility upgrade' is required for VS 2005, which of course you have to fork out for.  such a load of crap. 
anyway, what eventually worked for me was to go back into my VS 2005 dev machine and create a web set up project. go into the project properties and click the prerequisites button, select CR for .Net 2.0 and then just build the empty setup project. if you look in your output folder, there is an installation file called CRRedist2005_x86.msi.  Just whack this onto the server and your crystal reports should run fine, i didn't need to reboot or restart IIS.  Note there is no also need to copy any of the CR dlls to your web site bin folder. 

yet another miserable failure for crystal reports!

I picked up a tip on the Internet to display a "No records" message if the report contains no records, rather than leaving the user with an empty screen.  Simply Format the text field and enter a formula next to 'Suppress' with something like

Count({Table.Field}) > 0

This will suppress (hide) the message if there are records in the report.

I tried using a similar approach to hide a text field when there are no records, so it is essentially the same thing in reverse.  You would think a simple formula on the 'suppress' property this would achieve the desired effect:

Count({Table.Field}) = 0

But apparently the value returned by Count can be null if there are no records.  so you have to use:

IsNull(Count({Table.Field})) OR Count({Table.Field}) = 0

This is just another crystal reports annoyance, of which there are many.

i'm part of the dd/MM/yyyy world, and that often means running into problems when using software developed with MM/dd/yyyy defaults, such as crystal reports. 
i have all the report options set to use the system defaults, which are regionally set to ireland in windows, but that isn't enough, the dates still come out in MM/dd/yyyy format.  i have the date + date/time fields customised within crystal reports to dd/MM/yyyy but that isn't enough either.  with lots of hours googling and no answers that worked, i resorted to searching the registry and found that some of the user accounts were still using US regional settings.  If you look in HKEY_USERS > you see a list of all the account IDs on the computer.  My guess is that crystal reports must use the SYSTEM account or another non-interactive account, and it takes the regional settings from there.  So if you search for sShortDate in the registry, you will find all the appropriate settings and can replace the MM/dd/yyyy values with your preferred format.
i had to reboot to get it to take effect. 

For some reason i can't get SelectNodes to work with the IIS 6 MetaBase.xml file.  It just doesn't return any matches.  I think the structure may be non-standard or too complex, or else i'm not using it right.  but in any case, I worked around the problem by opening the file with File.OpenText and using Regex to parse out the nodes i'm interested in, namely the IISWebServer nodes.

string MetaBase = "";
using(StreamReader sr = File.OpenText(ConfigurationManager.AppSettings["MetaBasePath"].ToString()))
	MetaBase = sr.ReadToEnd();

foreach(Match m in new Regex(@"(?x:)<IIsWebServer[^>]*>.*?</IIsWebServer>", RegexOptions.IgnoreCase).Matches(MetaBase))
{
	XmlDocument doc = new XmlDocument();
	string IisCode = "", ServerBindings = "", ServerComment = "";
	doc.LoadXml(m.Groups[0].Value);
	XmlElement node = doc.DocumentElement;
	// now you can access the node attributes
...

since i installed .net 2.0 on my web server, i sometimes forget that you can't have .net 1.1 and .net 2.0 running in the same application pool, and then IIS kindly stops all the web sites if i make this mistake when configuring a new website. 
so i wanted to get an email about it if i forget, and went looking on d'internet for a decent little app to monitor events and email me if anything goes wrong. 
the best one i found is called Event Monitor, and it is a great little program.  nice enough UI and good filtering ability.

i set it up to monitor for events with id 1062 (the one that happens when i put .net 1.1 and 2.0 in the same app pool by accident..) and now i get an email straight away if it happens.  2 hours of tic-tac freshness in just 2 calories. great huh?!

i wanted to change the admin password on my web server through remote desktop, but Ctrl-Alt-Delete always goes to the local computer. 

i found out you can also use Ctrl-Alt-End to achieve the same thing, which works in remote desktop.

Thanks to Dusty Harper for his post on the server.networking MS newsgroup, to backup RRAS settings using Netsh:

Netsh Routing Dump > Routing.txt
Netsh RAS Dump > RAS.txt 

Then you can use Netsh Exec to playback the file.

Note: if you do a ntbackup of system state, the RRAS settings are also included in that.  i just like having the text file versions just in case.

MS Word crashed in the middle of a print job, and the document was stuck in the print queue, not obeying commands to delete or cancel it, even after reboots.
To fix this, i opened c:\windows\system32\spool\PRINTERS and deleted all the files there.  If the files are locked and won't delete, stop the Print Spooler service first and then delete the files.
This worked for me!

i encountered problems trying to shut down or reboot my windows server 2003 while logged in via remote desktop.  i initiate a shut down from the start up menu, and it logs me off and quits the RD session, but doesn't power off.  If i try and connect in again with RD, it shows me a blank screen on the server for a few seconds, and then quits with no error message.

if i try running "shutdown -s" then i click OK to the message that the computer is shutting down in 30 seconds, but it never does. system log entry: "Application popup: System Shutdown : The system is shutting down. Please save all work in progress and log off. Any unsaved changes will be lost. "

finally, if i push the power button it doesn't complete the shut down either. (i have 'shutdown' selected in power management for when the button is pressed).  i do see an event ID 26 in the system log at the time i pressed the button, with the message 'Application popup: Windows : Other people are logged on to this computer. Shutting down Windows might cause them to lose data. Do you want to continue shutting down?'.  this is a bit useless because the server has no mouse/keyboard/monitor, and i have no way to interact that message except using remote desktop (which doesn't even show that message on screen if i am logged in via RD), and i want to force a software shutdown without being asked questions i can't answer!

finally i found a work around to use the command line tool shutdown.exe with some more severe arguments.  This saves my raid array re-synching from hard-resets.

To Reboot:

shutdown.exe -r /t 10 /d p:1:1 /c "Maintenance, Planned"

To Shutdown:

shutdown.exe -s /t 10 /d p:1:1 /c "Maintenance, Planned"

IIS6 basic authentication (challenge/response) with PHP not working

HowTo: set up disk status monitoring (i.e. for a raid array) and send results by email

FIX: ISAPI Filter won't load on production server IIS

Howto: Disable windows automatically adding network printers and faxes

[FIX] Network bridge with Wireless + Wired connections is slow

A sample NTbackup script, with the current date/time in the filename and backup label

How to turn off Windows auto-expand (and search) for compressed folders

Problem opening web project in visual studio, due to mime-types on server.