So, yesterday I wrote an opinion-piece about Vista, and it appears that Bob Cringely did too. Completely unrelated, and I didn't read his piece before I wrote mine. Of course, his readership is more than mine. :-) His piece has generated lots of commentary - but I just flat out think he's wrong.

Granted, I'm not the average computer user. Not even the average power user. I've got physical machines that run XP, that run Vista, that run Server 2008, that run Server 2003 - and believe it or not, one that runs Linux. And probably twice as many virtuals as I've got physicals.

The market can't have it every way. Since XP was released, Microsoft has been absolutely PUMMELLED by spam, by viruses, by worms, by lack of hardware capabilities, by lack of software capabilities, etc. etc. etc. Microsoft responded to what the market demanded, and Vista is the answer.

Graphically, Vista is gorgeous - if you have the graphics horsepower to make it happen. Vista provides software support for technologies that weren't even conceived of when XP was released. The hardware support that Vista provides makes it MUCH easier for the OS to NOT crash when there are driver bugs. Or bugs in any add-on product. And on and on and on.

All of those things come at a cost - in memory and in processor.

If you want a minimal version of Vista - go install Server 2008. See how lean and mean it is. And how little it can do in the base configuration. Then, start adding the features and roles you require in order to get to a workable desktop machine, and see how those changes impact performance. In some ways, a desktop machine has to be more powerful than a server. It certainly has to have more "fluff".

I'm not a Microsoft "rah rah" man. However, I'm well aware of where I make my money - and that's based on Microsoft products. I criticize the Microsoft machine on a daily basis - and I do it in public forums, such as mailing lists, on my blog; and I do it in private forums, for betas (and even alphas) of certain software that I take a particular interest in.

Vista _IS_ sucky in some ways. And I've bugged those that affect me. For example, even after SP1, wireless doesn't "just work" like it did in XP. Many users have to reboot when switching wireless connections. For me, I'm tech savvy enough to open a command prompt and do an "ipconfig /renew". It's irritating.

But does that mean that Vista is going away? Don't be silly. Even if you hate Vista, it introduces many technologies that are part of the future of computing. You need to learn it. It's the stepping stone to what comes next.

Microsoft isn't abandoning Vista. They've made that clear too. Many people have taken the fact that there is so much talk about Windows 7 already to mean that Microsoft is abandoning Vista. The only reason that they can make THAT claim is because they choose to ignore that Microsoft has also stated that "never again" will there be 5+ years between operating system releases. It was simply too long, and Microsoft heard that message too.

You don't have to get with the program. But you should. Time marches on. And so does software - and hardware - and Microsoft.

Until next time...

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Posted by michael | 2 comment(s)
Filed under: , ,

What _IS_ the deal?

I practically got into an argument with another MVP recently over Vista...he said no one is buying it. I chose to disagree. He said no one is running it. I chose to disagree. By this point, he was pretty wound up...so I just suggested we agree to disagree....

According to Bill Gates, Vista has sold 140 Million licenses. That seems like a crap-load of Vista to me. Even if a third of those licenses are fall-backs to XP, it's still a crap-load of Vista.

OK, when Vista came out, there weren't a lot of drivers. No shocker there, the same was true when XP was released.

OK, when Vista came out, everything had been moved around and there was a learning curve. No shocker there, the same was true when XP was released.

OK, when Vista came out, there were bugs in the RTM code. No shocker there, the same was true when XP was released.

I mean REALLY. Come on. It's not like the software industry is brand new anymore. We all know how this works.

Major releases have problems at RTM. To ship is to choose; so the software isn't going to be bug free - but it'll work most of the time. It really truly may take a service pack for many configurations to work - the vendor has to see what people are using (if they aren't Apple) - there are more combinations and permutations "in the wild" than any vendor can test for during a beta process.

So....you don't like Vista. Fine. There are loads of new technologies present in it and you are obviously change resistant. Do all of them work 100%? Nope. But you can be VERY sure that they are there in Server 2008; and they'll be there in Windows 7 - plus more.

If you don't like it, provide constructive criticism. Saying "it sucks" does nothing for anyone. If you truly think everything about it sucks, then go Mac or try Linux on the desktop. There are some fine solutions there.

Until next time...

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Posted by michael | 1 comment(s)
Filed under: ,

Exchange 2007 SP1 RU2 is out! This one is pretty big for clusters and if you use POP3 and IMAP in your organization. I recommend you roll it out quickly.

More about it at KB 948016. Lots of patches in this one.

Posted by michael | with no comments
Filed under:

This is turning out to be SSL certificate week here at TheEssentialExchange.... not planned that way, just the interesting items that are crossing my computer. Yesterday, I wrote ISA 2006 and SAN/UC Certificates describing how ISA 2006 requires careful configuration in order to properly use a SAN/UC certificate.

Today, three more items you need to be aware of:

1) Windows Mobile (any version) does not support wildcard certificates

2) Outlook Anywhere (and RPC/HTTP for Exchange 2003) does not support wildcard certificates

3) UM requires that the machine name of the UM server be the first SAN (or only, for a singly named certificate) in the certificate

Lee Mackey, a fellow Exchange MVP, recently wrote: certs in exchange are like gouging your eye out with a spoon!

I echo his sentiment. That being said, there is no question that the product team understands that certificates are "too hard" right now in both Exchange Server 2007 and in OCS 2007. We'll just have to wait and see what they come up with to deal with this, in the future...

Until next time...

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Edited, May 10, 2008:

Wow, this brought a number of folks out of the woodwork to tell me how wrong I am!

1) Windows Mobile, as of version 6.x, DOES support wildcard certificates!

2) OA supports wildcard certificates if, in the MSSTD entry, you specify "*.example.com" INSTEAD of a specific hostname, such as "mail.example.com". Now, isn't THAT obvious? NOT!

3) No change to the UM requirement...

Thanks to my readers (and a Microsoft employee for item (2))...

Michael B.

Posted by michael | 2 comment(s)
Filed under: ,

Much has been written about ISA 2006 "not supporting SAN certificates".

And, to some degree, this is true: the RTM version of ISA 2006 will only recognize the first subject alternative name in a certificate.

So what do you do if the name you need to use is not the first SAN in the certificate? You fake it out. :-)

What you publish using ISA 2006 is the entire certificate, to the outside. What's important to ISA is how you use it, inside. Clear as mud?

This is the three-step work-around:

  1. For the rule that you have an issue with, open the Property sheet.
  2. On the "Web Farm" tab, the "Internal site name" must be the first subject alternative name published on the certificate.
  3. On the "Public Name" tab, add both names - the name you want to use (e.g., mail.contoso.com) as well as the first SAN on the certificate (e.g., autodiscover.contoso.com).

That's all it takes. And, of course, if you add the first SAN first when you create the certificate, none of this is a problem either. And most Certification Authorities will re-issue certificates within a few days for free, if you messed the request up.

Truly, much ado about little...

Until next time...

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Posted by michael | 4 comment(s)
Filed under: ,

When Exchange Server 2007 was first released (even before then, if truth be told), there was quite an outrage that in order to use a simplified configuration process for Outlook Anywhere, OWA, and AutoDiscovery, the Exchange administrator had to use a special kind of SSL certiifcate that supports multiple Subject Alternative Names (SANs).

Now, most Exchange administrators had never heard of such a thing (I know that I hadn't).

A SAN certificate falls "in the middle" between a wildcard certificate (*.example.com) and a single-server certificate (www.example.com); including a feature that wildcard certificates cannot support - different domains in the alternative names. This allows a SAN certificate to have a list like this: mail.example.com, owa.example.com, autodiscover.example.com, mail.example.local, owa.example.local, and autodiscover.sample.local. Where the first three are EXTERNAL server names, and the last three are INTERNAL server names. Pretty slick, eh?

The problem was the cost. SAN certificates are MUCH more expensive than a single name cert. (Supply and demand, of course...)

Then, Microsoft began using SAN Certs in OCS 2007. The common name for a SAN certificate is gradually changing over to UC Certificate (for Unified Communications Certificate).

Microsoft only has three "certified" partners for providing UC certificates. They are presented in KB 929395 with prices from $325 to $599 per year for a UC certificate. Compare this to RapidSSL at $20 per year for a standard SSL certificate....(all prices in US dollars).

So, as competitive forces usually do, someone else came up with a less expensive solution.

Certificates for Exchange produces inexpensve UC certificates (starting at $30 per year) that work 100% with Exchange Server 2007 and all version of Windows Mobile 5 and up.

I am not associated with Certificates for Exchange in any way. I'm just happy that someone released an inexpensive UC certificate solution and thought you, my readers, would be too.

Until next time...

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Posted by michael | with no comments
Filed under: ,

During a database backup you got a dreaded -1018, or -1019 error - indicating that your Exchange store is corrupt. But Exchange keeps running...

Or, you had kept dozens of "old" user mailboxes sitting around and you finally decided to purge them. After that, you find your Exchange database is half empty.

What do you do in these situations?

In the "old days", you would take your Exchange store offline, take an offline backup, and then start up ESEUTIL. The first step is "garbage collection". This removes all the unused space from the Exchange database, and compresses all the indexes - giving you a smaller Exchange database. Then you remount the database and it's all good.

Note: Garbage collection only removes unused space that has been processed by "online maintenance". If you remove 10 GB of mailboxes, and don't allow online maintenance to complete first, when you run garbage collection, that space will not be returned - because as far as the database software is concerned, the space is still in-use. This is similar to the difference between soft-delete and hard-delete in individual mailboxes.

Next, you take an online backup. The act of doing a garbage collection invalidates your earlier backups and transaction logfiles.

If you started this process because of a -1018 or a -1019, and the backup completes successfully, you are golden! You've successfully returned your database to production quality with no data loss. If you again receive an error, you must execute a database repair, during which you will almost certainly lose data. I'm not going to cover that here; if you are not certain what to do, your best bet is to open a case with Microsoft CSS/PSS.

Now, depending on your database size, and the amount of whitespace in that database, you may have had your mailbox database offline for many hours - causing pain and heartache for your users, and potentially bounced e-mail.

Let's talk about a mechanism for avoiding this downtime.

Note: This process will only apply to Exchange 2000 Server and Exchange Server 2003 Enterprise Editions, or to Exchange Server 2007. In Exchange 2000/2003, Standard Edition only supports a single mailbox store. In Exchange Server 2007 Standard Edition that limitation was (finally!) increased to five mailbox stores.

First, you must verify that you have at least twice as much disk storage as the mailbox store in question. If you do not, then an easy way to get significant additional temporary storage is to attach a RAID-1 (mirrored) USB disk (a couple of options are from Lacie's BiggerDisk line and Netgear's StorageCenter line - both of which I have used before). Please don't use a non-RAID solution. Cheap disk tends to not be enterprise quality, and fails quickly. Only trust your mailbox data to a RAID-based disk solution.

Next, create a new storage group and a new mailbox store in the location where you have the extra disk space.

Next, use the move-mailbox wizard to move all of the mailboxes from the old mailbox database to the new mailbox database.

Next, dismount the old mailbox store.

Next, REMOVE THE OLD MAILBOX STORE FILES. Yep, you read that right. But ENSURE and VERIFY you are removing the correct files. If you want to make certain, rename them instead and remove them later.

Next, remount the "old" mailbox store. A new empty database will be created.

Next, use the move-mailbox wizard to move all of the mailboxes from the temporary mailbox database back to their original mailbox database.

Next, dismount and remove the temporary mailbox database and storage group.

Finally, take a full backup.

There is no system downtime. During the entire process your Exchange server can continue to send and receive e-mail. However, there is some impact. Those mailboxes that are currently in use when you try to move-mailbox will not move. Outlook cannot connect to a mailbox in the process of being moved. However, this is a minimal impact compared to hours of system downtime. And, you can do it all in the background.

Add this to your repertoire. Practice it. It's easy to do. I've told you how. :-)

Until next time...

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Posted by michael | 2 comment(s)
Filed under:

A question came up recently on a mailing list I hang around. A poster wrote: "I'm looking for a concise explanation of when/how the transaction logs are committed.  Specifically, is there another way to force logs to be committed other than the backup or stopping the IS?  (E2K3)"

As with many things in Exchange, when you get down to the bits and the bytes, things are not always as they seem. My response, which some of you might be interested in, went more-or-less less like the below (I've expanded it a little, and added a few paragraphs on checkpoint exhaustion). Also note that this only applies to streaming backups. Shadow-copy backups are different.

A concise explanation? Hrmmm.

Well, first, doing a backup does NOT force logs to be committed. It means that the backup process waits until a checkpoint occurs, flushes the current transaction log, and during the backup additional checkpoints are not allowed to occur (that is, nothing is allowed to be flushed to the ESE database until after the backup is complete – leading to an increasing checkpoint depth, and in extreme situations, checkpoint exhaustion – discussed a little later). The ESE buffers are not flushed. The "checkpoint depth" is how many checkpointed logfiles must be processed before the ESE database is current with the transaction log files.

So...in the normal case, data is written in a serialized fashion to the in-memory ESE cache and to the log buffers, as updates occur in an Exchange database. Logs are written to disk as logs fill up, or as transaction commits occur. (This may mean that a log can have nothing but a checkpoint record in it – but that is not the normal case.) In the ESE buffers, I/O is accumulated and prioritized by a process known as the “lazy writer”. The lazy writer scans the ESE buffers for dirty (modified) pages and builds an optimized list to flush those to disk. As that list is flushed to disk, the pages are marked as “clean”, and the checkpoint is marked as having advanced (on a transaction by transaction basis, not a log by log basis). Whenever the transaction in a log are fully advanced, then the checkpoint file and the database header are updated.

During a backup, the lazy writer is paused.

The ONLY time you are assured that logs are fully committed is during clean shutdown, or after running soft recovery.

Now, consider the situation when a backup is in process and it gets hung (i.e., waiting on a tape, or there was a fatal write error on the tape and the backup application crashed). What happens? There are three possible answers.

1) The backup application caught the exception (i.e., the error that caused the crash) and cleaned up after itself. In this case, it appears as if the bad backup never happened and everything just goes back to normal. This is the best possible case.

2) The backup application didn't catch the exception, but "notices" when it next runs that there was a backup in process and isn't one now, but the backup state is still "dirty". Then, the backup application cleans up the old "dirty" backup state, and proceeds as normal. This is the middle of the road case.

3) The backup application didn't catch the exception, and when it goes to start the next backup, it gets an error from Exchange - "backup already in process". Then, because of that error, the backup aborts. This is the worst situation. To clear the "backup in process" flag requires a restart of the Microsoft Exchange Information Store service.

In any of these cases, the checkpoint depth is continuing to advance - and the log files are accumulating, but nothing is being committed to the database. This is a problem! Eventually, the Information Store will say "enough!" and dismount the storage group. That's right - it will dismount the entire storage group.

This is called "Checkpoint depth exhaustion". After 1,008 logfiles have been generated,  Exchange automatically shuts down the storage group. There is an implementation limit of 1,024 logfiles in a checkpoint, and this shutdown prevents the Exchange database from being corrupted.

After the storage group is dismounted, MSExchangeIS can simply be restarted and the storage group remounted (this will cause all of those transaction logfiles to be replayed - it may take quite awhile!).

This can also happen in very very high update situations (such as running many move mailbox operations all at once time).

Your best bet is to monitor for this. If you are using OpsMgr 2007, this is already one of the issues checked for. If you want to add this check manually, add the performance counter "Database ==> Instances\Log Generation Checkpoint Depth".

For more information on this issue, see KB 905801 and KB 819771.

Until next time...

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Posted by michael | 2 comment(s)
Filed under: ,

I hate to sound like a marketing goon, but I have to tell you about Xobni.

But I've been using Xobni for four days. I don't know how I ever got by without it. Comparatively speaking, I can now process my e-mail at lightspeed.

I don't know how YOU work with e-mail, but I spend a great deal of time referring back and dealing with the same people and pulling up prior e-mails, etc. etc. In order to do that, I've developed a pretty well-organized but moderately complex filing system based (of course) on Outlook folders.

With Xobni, I still file everything, but Xobni keeps it organized for me. If I click on an e-mail from someone, Xobni finds me all related threads, all related attachments, contact information for that party, and other stuff.

And yes, Xobni is "inbox" spelled backwards.

The website is the obvious place: http://www.xobni.com.

Until next time...

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Posted by michael | with no comments
Filed under:

Yesterday (Thursday, 2008-04-17) was the last day of the 2008 MVP Summit. Over one-half of the MVPs attending were from countries other than the United States. It was interesting to hear the dozens of languages being spoken around me this week.

Very little of what was covered during the Summit is not covered by NDA. But you can be assured that MVPs from every product that Microsoft produces were there learning about what is coming and providing "feedback" to the Microsoft product teams. Feedback can get quite .... loud .... at times. MVPs tend to be quite passionate about their products of interest and very protective of their user communities. Many features present in current releases of Exchange came at MVP "encouragement".

In fact, one feature added quite recently was only added because MVPs said "it has to be there". And in the future, we see that it will be expanded significantly to meet other needs. Yay MVPs!

The last presentation of the Summit was a keynote given by Steve Ballmer. During his hour on the stage, he discussed many things and took about 20 minutes of questions from the audience. Two of the items he said really struck me. The first: "Windows Vista...is a work in progress". The second was: "We can't go five years in between releasing versions of the operating system".

There are many things that can be read into those two statements. However, it is clear that Microsoft knows that Vista isn't where it should be yet, and it's likely that "Windows 7" won't take as long to get here as Vista did.

Changing directions...Steve took several questions about Groove vs. SharePoint as did Ray Ozzie, who spoke before Steve. There is a definite belief in those MVP communities that one of these collaboration tools has a limited lifetime (there is a lot of overlap). Who knows what it will be?

You can read more about Ballmer's speech in an article from the Seattle Times.

As always I met many people, shook many hands, got to see some old friends, and lifted a few while bonding with the other geeks around me. :-) A good time and very worthwhile.

Until next time...

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Posted by michael | with no comments
Filed under: , ,

Consider the format of an Exchange database.

It's flat. Very flat.

Take a mailbox. Basically, it is a table that contains pointers to other tables. What are those other tables? Well, your Inbox, Sent Items, Calendar, Tasks, Notes, etc....

Every folder is a table. Each table contains objects and pointers to other tables. An object may be a message, a task, a calendar entry - basically the low-level objects which Outlook manipulates.

Every view is an index - and an index is just a table of sorted pointers to objects (plus a small bit of meta-data).

So... each mailbox may have thousands of tables. A mailbox database may have millions of tables. Yes, you read that right: millions.

Why is it done that way? Easy answer! To allow the user maximum flexibility. The end-user is not limited in the number of views, not limited by what he can sort by, how many objects you can contain in a folder (although there certainly are performance issues associated large numbers of items), etc. etc. Exchange Server generates these views on the fly whenever Outlook requests them. Real-time. The end-user is not arbitrarily limited in any significant way.

Give a great pat on the back to Exchange!

How does Exchange do this? With ESE. Because ESE - the Exchange database engine - supports these features.

Now, consider Microsoft SQL. A great DB engine. However, it doesn't really generate tables on the fly. Or views on the fly. Or have a hierarchical mechanism for storing table trees. Or clean up after itself. All of these functions have latency in SQL...

So....what would have to happen? A complete redesign. Not just a simple "let's use a different backend". And, it's likely that a lot of features would be lost in any such translation.

Do I want Exchange in SQL? No. Do I think it would work well? Not with any SQL Server released to date.

In the future? Who knows? Maybe. But I don't know.

Until next time...

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Posted by michael | 2 comment(s)
Filed under:

I and 4,000 MVPs from around the world have accumulated in Seattle/Redmond this week. Where do we all go? To the bar, of course!

Last night, a few Exchange MVPs and the odd PowerShell MVP were sitting around discussing stuff I can't talk about here, when another MVP came to join us (who happens to work for a major storage vendor). Hey, we are open minded, no problem there!

Then, out of nowhere, he began to bash our beloved Exchange!

Interestingly enough, the basic complaint was this: Exchange degrades too gracefully. That is, you can configure Exchange in ways that are downright stupid, it will let you, and it will "sorta" run - until you finally hit that "straw that breaks the camel's back" and then it crashes.

Well, that's a valid complaint. Exchange goes to extreme lengths to allow your e-mail to go through - because e-mail is mission critical. This is what it should do, in my opinion.

And in doing this, if you want to put 1.2 million mailboxes on a single LUN of a SAN; well, it'll let you. Doesn't mean that it is the right thing to do! And it doesn't mean that Exchange is broken because it lets you do this!

There is nothing that excuses poor configuration. Whether it is Exchange itself, the storage subsystem, the network, whatever. The entire Exchange ecosystem, in large installations, must be planned and monitored.

Truthfully, in many small to medium installations, Exchange is so forgiving that you can do almost anything to it, and it'll continue to run. Most SMORGS don't have access to the technical expertise to do a proper design/rollout/support/etc. and with the hardware of today, that's ok. And Microsoft puts together packages of software to help this be OK (think Small Business Server and Essential Business Server).

But when an installation gets large than, oh, around 250 mailboxes - you best begin to take planning seriously. Planning, monitoring, and proper operations are a system administrator's ongoing responsibility.

Until next time...

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Posted by michael | with no comments

Even though I get pretty hot under-the-collar sometimes when Microsoft removes for-free functionality (Exchange backups on Windows Server 2008, for example), there are some great tools that Microsoft provides for-free that are just as good as their for-pay alternatives - and more tightly integrated with other Windows functionality. However, they are often not well known.

One of those is WDS - Windows Deployment Services. WDS provides easy/simpler ways to, well, Deploy Windows! (D'Oh!)

Part of WDS is ImageX. ImageX provides a free-and-easy way to create images of Windows desktops, including deployment images. One of the great features of ImageX is that you can easily load drivers, patches, service packs, etc - AFTER the image is created, without creating a brand new image. This is a great time saver. It also allows you to create and install "generic" images that have multiple HALs (Hardware Abstraction Layers) and storage subsystems (as long as the drivers are included in the base image!).

Check it out.

Windows Automated Installation Kit

ImageX Technical Reference

Until next time...

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Posted by michael | with no comments
Filed under: ,

Recently on a mailing list, several folks expressed some confusion about the scaling that Perfmon (Performance Monitor) does in its graphs and how it affects the data.

The answer is: it doesn't. The scaling is used only to make the graph fit in the Perfmon window. The values contained in the Last/Average/Minimum fields are not scaled, they are the actual counter values.

If you want to prove this to yourself in another way, click on "Change Graph Type" (ctrl-G for you keyboard folks like me) and select "Report". You'll see the same size of values reported in that list.

If you've never played with Perfmon before (or the entire "Reliability and Performance Monitor" that is part of Vista and Server 2008), you should learn how to use it. It can make debugging performance issues a lot easier.

Until next time...

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Posted by michael | with no comments

I wrote and supported two versions of a Catchall Mailbox script that worked with Exchange 2000 Server and Exchange Server 2003. Version 2 of it is available on this blog, at this link: Exchange 200x Catchall Script, Version 2.

However, the transport engine in Exchange Server 2007 is completely different. The old script interfaces no longer work.

However, as a sample, an Exchange engineer developed the Catchall for Exchange Server 2007 and posted it on CodePlex, calling it CatchAllAgent Exchange 2007 Transport Protocol Agent. If you need the functionality for Exchange Server 2007, that's where to get it!

Until next time...

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Posted by michael | 2 comment(s)
Filed under:
More Posts Next page »