Exchange 2007 Backup Woes

We use Exchange 2007 and backup using CommVault Simpana 8.0. We had an issue with a store that ran out of disk space, and then dismounted. The store logs were moved to another drive and the store mounted again. From this point on, backups (full or incremental) failed with CommVault with the following error:
Error Code: [28:163] Description: The Exchange server returned the following error message: [0x50d] - [Unknown]. Api: [HrESEBackupSetup()], Item: [Fourth Storage Group]

NTBackup could backup the store but took 20 hours compared to ComMVaults just over 2.

Investigating the event log produced an event id 9840 with error 1293:
An attempt to prepare the storage group 'Fourth Storage Group' for backup failed because the storage group is already in the process of being backed up. The error code is 1293. (Note that if a backup was recently aborted, then it may take several minutes for the system to detect the aborted backup and initiate backup cleanup procedures, so this message may be generated if an attempt was made to backup a storage group before a previous backup attempt had fully terminated.)

Turns out, the store filled during a backup leaving the backup in progress flag set. NTBackup works because it uses the old Exchange backup API's, not the VSS backup used by CommVault.

The fix was to dismount and then mount the affected store. 

Transistion to Cloud Servers

I'm moving all my locally hosted content to cloud based servers. So far I have moved my email to Google Apps For Your Domain and I have created a new blog (no content yet) to replace this one.

The Gmail move went pretty well as all I had to do was update MX, SPF and some CNAME records. The only non-working part was the start page as Google seem to have dropped support for this. I had actually setup apps before this limitation was implied, and I can activate the start page but it just doesn't work. There are others who mention this and the work-round seems just to use a Google Sites page if you want this. In fact there's quite a complete template for sites, just for this. I don't really need it anyways.

The blog will move slower. I have moved it off to a smaller sevrer for now and I'm trying to find a way to export the posts from CommunityServer to Blogger. I may just have to use the smaller server to setup redirects to the new blog.

Creating a dynamic distribution group with advanced filtering on Exchange 2007

In Exchange 2003, it was very easy to build a dynamic distribution group with the GUI. Exchange 2007 is much more limited unless you use Powershell. I needed to create a dynamic distribution group targeting a particular server, as long as the user mailboxes didn't have a forwarding address. The reason for wanting to avoid the mailboxes with forwarding addresses is we still have Quest's Exchange Migration Wizard creating forwarding details for users on the legacy Exchange system.

The process is a two part process:

• Create the dynamic distribution group
• Set the filter criteria for the distribution group

To create the dynamic user, type the following all on one line in the Exchange Powershell console (don't type in the bold italic comments):

New-DynamicDistributionGroup 'The applett

-Name "Group name" 'What the group should be called

-IncludedRecipients MailboxUsers 'Just include users with mailboxes

-OrganizationalUnit "domain.com/ou1/ou2" 'Where to create the dynamic distribution group
e.g. New-DynamicDistributionGroup -Name "Group name" -IncludedRecipients MailboxUsers -OrganizationalUnit "domain.com/ou1/ou2"

To set the filter:

Set-DynamicDistributionGroup 'The applett

"Group Name" 'What the group is called

-Notes "Blah blah blah" 'Description

-RecipientFilter {ServerName -eq 'ServerX' -and ForwardingAddress -eq $Null} 'Mailbox exists on ServerX and the forwarding address is empty

-RecipientContainer "domain.com/ou1/ou2" 'What container the recipients are under - It can just be the domain

e.g. Set-DynamicDistributionGroup "Group Name" -Notes "Blah blah blah" -RecipientFilter {ServerName -eq 'ServerX' -and ForwardingAddress -eq $Null} -RecipientContainer "domain.com/ou1/ou2"

Once set with Powershell, they can be viewed by the GUI, but not changed. Beware that the preview button does not take account of the -RecipientContainer parameter - It will show you all users in the domain that match.

Inspiration from:

http://technet.microsoft.com/en-us/library/bb123796.aspx

http://technet.microsoft.com/en-us/library/bb738157.aspx

DanW

How-To Prepare A Certificate With Subject Alternative Names Using Exchange Powershell

1. Use powershell from an Exchange 2007 box to generate a certificate request (all on one line and replace italics with your details):

New-ExchangeCertificate
-GenerateRequest:$true
-Path C:\NewCert.req
-DomainName fqdn1.domain.tld,fqdn1,fqdn2.domain.tld,fqdn2
-PrivateKeyExportable:$true
-FriendlyName "fqdn1.domain.tld"
-IncludeAcceptedDomains:$False
-Force:$true
-SubjectName "CN=fqdn1.domain.tld,OU=Department,O=Organisation,L=Location,S=County,C=GB"

2. Request a certificate from your AD certificate server, (http://yourcertserver/certsrv). This needs to be an advanced request, using the request file generated above, and for a web server. Download it to your local drive as C:\CertNew.cer.

3. Import the certificate into your Exchange server using:

Import-ExchangeCertificate C:\NewCert.req

4. On the Exchange server, use the MMC certificate snap-in, export the certificate. Export it with the private key to a PFX file. Make sure you target the computer certficate store when loading the MMC. Use a password to protect the exported certificate.

5. On the target server, load the certificate MMC, again targeting the computer store and import the pfx file.

6. Using the IIS console, switch your webserver to the new certificate.

7. You can delete the original certificate from your Exchange server if you wish.

Isinteg fails to run on Exchange 2007 with Error: FULLCHKMGR::EcReadRowCountGlobalFlag failed with error JET_wrnColumnNull

I had a corrupted mailbox, so created a temporary store to move the mailbox into. I then wanted to run an isinteg on this store before returning the mailbox back to the original store.

The first part went ok, but when I went to run isinteg I received the following error in the logfile:
Error: FULLCHKMGR::EcReadRowCountGlobalFlag failed with error JET_wrnColumnNull

Searching around prodeuced a few solutions, but none in English! I found the following post:
http://www.spurius.nl/?p=46

My Dutch isn't very good, but I surmised that the command to run was:
isinteg -s servername -fix -test rowcounts

I ran this, and indeed I could then run an isinteg with -test alltests. Thanks to whoever that blogger is!

Update Rollup 3 for Exchange Server 2007 Service Pack 1

This (KB949870), was applied by Windows Update last night, and I woke up to no incoming email. On investigation, I found that there was no smtp (25) port open, and that the Microsoft Exchange Transport hadn't started after the reboot.

I attempted to manually start the service, but no joy as it timed out after 30 seconds. I reviewed the event logs, nothing there, tried again and it worked. Did a trawl back through the event logs when the server restarted, and again nothing. There were some eventid 3001\3003 from the EvntAgnt (log file not at end) so I may be missing some logs.

The only other issue I can think of, is that this is a single server Exchange 2007 installation - I.e. The transport service is running on the same server as the mailboxes rather than on a separate edge server. I would think this receives no testing by Microsoft!

I will have to monitor this the next time I reboot. Is anyone else having this issue?

DPM 2007 DPMRA Error 10048 (0x2740)

After rebooting my Exchange server because of DNS issue, the DPM protection stopped. Further investigation found this in the event log:
The DPMRA service terminated with service-specific error 10048 (0x2740).

Google found this:
http://support.microsoft.com/kb/947682

Essentialy you use 'netstat -oan' to find the PID of the process using port 5718 or 5719, which are both required by DPMRA.
Then use 'tasklist' and 'tasklist /svc' to discover which process it is. In my case, it was the MS Exchange Information Store.

I fixed it by stopping all Exchange services, starting DPMRA and then restarting Exchange.

It's kind of weird that MS Exchange can use the DPMRA ports when DPM is speciffically for protecting Exchange (amongst other things).

How-to Enable antispam updates on a single Exchange 2007 server

There is a handy little script that you can run from the Exchange Management Shell:
InstallAntispamAgents.ps1

To update manually:
Get-AntspamUpdates

To update automatically:
Enable-AntispamUpdates -UpdateMode Automatic

How-to Allow a distribution group to receive mail from the internet

Found the answer here:
http://exchangepedia.com/blog/2007/07/new-distribution-groups-do-not-receive_31.html

Use the following command in Exchange management shell:
Set-DistributionGroup "Group Name" -RequireSenderAuthenticationEnabled $false

How-to Create antispam reports

There are a number of reports avaiable in Exchange Management Shell.

You must first change into the scripts directory (cd C:\Program Files\Microsoft\Exchange Server\Scripts) and then try the following:
get-AntispamFilteringReport.ps1
get-AntispamSCLHistogram.ps1
get-AntispamTopRecipients.ps1
get-AntispamTopRBLProviders.ps1
get-AntispamTopBlockedSenders.ps1
get-AntispamTopBlockedSenderDomains.ps1

How-to grant Sendas for Distribution Groups

Found the answer here:
http://exchangepedia.com/blog/2007/03/how-to-assign-sendas-right-using.html

Run the following in Exchange Management shell:
Add-AdPermission "Distribution Group Name" -user "Mailbox Name" -AccessRights extendedright -ExtendedRights "send as"

How-to Backup Exchange 2007 on Server 2008

After all the hassle of getting Exchange 2007 setup yesterday, I found that I had no way to backup my Exchange 2007 server running on Server 2008. For whatever reason, Microsoft do not supply NTBackup and there is no in-built way of backing up Exchange 2007 when run on Server 2008.

Now, I could opt for a third party backup program or use Microsoft DPM, but for my 4 user network, this is a little bit of an overkill.

After some research, I found this article on how to install NTNackup on Vista:
http://www.petri.co.il/installing_windows_xp_ntbackup_on_windows_vista.htm

I followed this by copying the following binaries from my old server (C:\Windows\System32\) to my new server (C:\Program Files\NTBackup\):

• ntbackup.exe
• ntmsapi.dll
• vssapi.dll

I installed the removable storage manager from server manager:

• Start Server Manager
• Click Features
• Click Add Features
• Select Removable Storage Manager
• Click Install

When NTBackup starts, you will see an error regarding removable storage, but I just ignored this and clicked "Do not show this again"

Upon running NTBackup, I could see drives and system state, but no Exchange!

I then found this:
http://support.microsoft.com/kb/275876

I copied esebcli2.dll from my old server (C:\Program Files\Exchsrvr\Bin) to my new sever (C:\Program Files\NTBackup)

I ran regedit and changed the esebcli2 key from C:\Program Files\Micrsoft\Exchange Server\Bin\esebcli2.dll to C:\Program Files\NTBackup\esebcli2.dll.

I ran NTBackup and backed up my Exchange store to disk!

Exchange 2007 on Server 2008

What a day!

I've learnt a hard lesson today. Never unbind IPv6 from Server 2008 if its running Exchange 2007.

I had some problems adding some Vista laptops to a Server 2008 domain today. I couldn't determine if it was SP1 RC1, UAC being turned off or IPv6. The computers would not join the domain using the GUI. I had to manually extract the XP SP2 version of netdom and use that, after manually creating the computer accounts in the domain.

All was well until I restarted the Server. The Exchange Transport Service hung in "starting" and I could not connect to the Exchange server using the Power Shell or Console. I kept getting AD errors from these tools, and lots of errors about the Exchange AD Topology service not being able to contact AD.

To cut a long story short, I flattened the server and started again - Only to discover that IPv6 wasn't the problems with the workstations joining the domain, but it was the problem for Exchange 2007!

I never did get to the bottom of the problem joining the domain though. At least I have a work round now - Use XP SP2 Netdom!

Problems installing and using Exchange 2007

I'm upgrading my Exchange 2003 organisation to Exchange 2007 this weekend, as a reinforced learning experience for when I begin to deploy Exchange 2007 for a major customer.

I had a couple of problems:

1. A setup error occured: "[ERROR] Access to the path 'D:\Setup\ServerRoles\Common\64' is denied"
2. When I re-ran setup, it installed, but there were no routing group connectors between the old routing group "First Administrative Group\First Routing Group" and the new routing group "Exchange Administrative Group (FYDIBOHF23SPDLT)\Exchange Routing Group (DWBGZMFD01QNBJR)".
3. I created the routing group connectors, and moved a mailbox, but mail would not flow. Kept getting 5.7.1 Relay Denied errors from the new organisation.

I found a few references:

Setup error:
http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=2746265&SiteID=17
http://support.microsoft.com/kb/943507/en-us

How to install Exchange 2007 into an Exchange 2003 organisation:
http://technet.microsoft.com/en-us/library/bb124350(EXCHG.80).aspx

How to remove Exchange 2007 roles:
http://technet.microsoft.com/en-us/library/bb124115(EXCHG.80).aspx

For some reason, my new Action Pack came with Exchange 2007 SP0 and not SP1.
I've downloaded Exchange 2007 SP1 and will reinstall this when I have removed Exchange 2007 from the server.

Exchange 2007 SP1:
http://www.microsoft.com/downloads/details.aspx?FamilyId=44C66AD6-F185-4A1D-A9AB-473C1188954C&displaylang=en

Exchange 2007 SP1 Release Notes:
http://www.microsoft.com/downloads/details.aspx?FamilyId=5770BD59-376E-42EC-B940-BE6225CD97FF&displaylang=en


NB:Just what does FYDIBOHF23SPDLT and DWBGZMFD01QNBJR mean?They are shifts of "EXCHANGE12ROCKS". First one, add 1 to each character, second one, deduct 1 from each character.

How to Display SCL (Spam Confidence Level) in Outlook 2007

Firstly, download and install this Exchange Event Sink as described here:
http://www.ivasoft.biz/showscl.shtml

Secondly, create a custom formula field to display the SCL as described here:
http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=739718&SiteID=17

The use of Right([Comments],5,2) allows an SCL of -1 through 9 to display correctly.

Of course, if you are using Outlook 2007 with Exchange 2007 you can see the Spam score by looking at the Internet headers - I have not seen a method to display this on the Outlook email list page yet.
http://technet.microsoft.com/en-us/library/bb124595.aspx

Downgrade Exchange 2007 B2 to Exchange 2003 SP2

Oh, what fun and games I had.

First I backed up all the mail:

• Outlook export to PST
• ExMerge (Hey - It worked on Exchange 2007!)
• NT Backup of store

Then I removed Exchange 2007 and reininstalled Exchange 2003 - Except it didn't quite go that smooth,

The installation errored saying that it could not install the information store and to check the install log. I found this error message:
mode = 'Install' (61953) CBaseAtom::ScSetup (f:\titanium\admin\src\udog\setupbase\basecomp\baseatom.cxx:842)
Error code 0XC00736B1 (14001): This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem.

Tracing back up the install log found this message:
Interpreting line -- ID:31259 --
CInsParser::ScProcessLine (f:\titanium\admin\src\libs\exsetup\hiddenw1.cxx:1225)
Error code 0XC00736B1 (14001): This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem.

Tracing back futher led to this line:
Copying c:\program files\common files\microsoft shared\cdo\cdoex.dll
Preserving existing copy of c:\program files\common files\microsoft shared\cdo\cdoex.dll
Sure enough, typing "regsvr32 c:\program files\common files\microsoft shared\cdo\cdoex.dll" produced the same error code.

I deleted the copy of cdoex.dll and replaced it with the copy from the Exchange Server 2003 CD and re-ran the regsvr32 command - It worked. I clicked "Retry" and the install completed.

I did receive an error message about the wrong version of mapi32.dll being on the hard disk so I also replaced from the CD too.

Applied Exchange 2003 SP2 and visited Microsoft update to apply some hot fixes and everything appears to be working well.

Exchange Server 2007 Beta2 Maximum Mail Stores

I was trying to work out the limits of the current beta of Exchange 2007 in terms of storage groups and mailbox stores and through experimentation I discovered that they are:

• Maximum Storage Groups: 50
• Maximum Mail Stores per Storage Group: 5

I guess that's a grand total of 250 mailbox stores then. That's a massive improvement over Exchange 2003!

Grant full mailbox permissions in Exchange 2007 Beta2

Final step upon upgrading the test server to Exchange 2007 Beta2 was to give me full access to all the mailboxes. It's another Power Shell task and I managed to find the syntax at the Microsoft Exchange Team Blog.

Add-MailboxPermission mailbox -AccessRights FullAccess -user user

This gives user full access permisions to mailbox.

Exchange 2007 Beta2 - Allow hub connector to receive unauthenticated SMTP mail

You need to set the permissions to allow Anonymous users to submit mail. This is not in the GUI so you must use the Power Shell.

1. Set the permission:
set-receiveconnector 'Default CHEF' -PermissionGroups:"ExchangeUsers,ExchangeServers,ExchangeLegacyServers,AnonymousUsers"Replace CHEF with your Exchange server name!

2. Verify the permission:
get-receiveconnector | format-listSchema: Microsoft.Exchange.Data.Directory.SystemConfiguration.ReceiveConnectorSchema
AuthMechanism: Tls, BasicAuth, BasicAuthPlusTls, ExchangeServer
Banner:
BinaryMimeEnabled: True
Bindings: {0.0.0.0:25}
ChunkingEnabled: True
DefaultDomain:
DeliveryStatusNotificationEnabled: True
EightBitMimeEnabled: True
EnhancedStatusCodesEnabled: True
ExternallySecuredAsPartnerDomain:
Fqdn: mail.thefoleyhouse.co.uk
Comment:
Enabled: True
ConnectionTimeout: 00:10:00
ConnectionInactivityTimeout: 00:05:00
MessageRateLimit: unlimited
MaxInboundConnection: 5000
MaxInboundConnectionPerSource: unlimited
MaxInboundConnectionPercentagePerSource: 100
MaxHeaderSize: 64KB
MaxHopCount: 30
MaxLocalHopCount: 3
MaxLogonFailures: 3
MaxMessageSize: 10MB
MaxProtocolErrors: 5
MaxRecipientsPerMessage: 5000
PermissionGroups: AnonymousUsers, ExchangeUsers, ExchangeServers, ExchangeLegacyServers, CustomPipeliningEnabled: True
ProtocolLoggingLevel: None
RemoteIPRanges: {0.0.0.0-255.255.255.255}
RequireEHLODomain: False
RequireTLS: False
Server: CHEF
SizeEnabled: True
TarpitInterval: 00:00:05
AdminDisplayName:
ObjectCategoryName: msExchSmtpReceiveConnector
ExchangeVersion: 0.1 (8.0.535.0)
CurrentObjectVersion: 0.1 (8.0.535.0)
Name: Default CHEF
DistinguishedName: CN=Default CHEF,CN=SMTP Receive Connectors,CN=Protocols,CN=CHEF,CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=The Foley House,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=thefoleyhouse,DC=ad
Identity: CHEF\Default CHEF
Guid: 0675a1d2-9c19-4068-af33-782d1c27f562
ObjectCategory: thefoleyhouse.ad/Configuration/Schema/ms-Exch-Smtp-Receive-Connector
ObjectClass: {top, msExchSmtpReceiveConnector}
OriginalId: CHEF\Default CHEF
WhenChanged: 01/10/2006 19:58:31
WhenCreated: 30/09/2006 00:19:26
ObjectState: Unchanged
OriginatingServer: chef.thefoleyhouse.ad
IsReadOnly: False
Id: CHEF\Default CHEF
IsValid: True

How to set recipent email addresses in Exchange 2007 Beta2

To set recipient email address policy, do the following:

1. In the Exchange Management Console, navigate to 'Organization Configuration' > 'Hub Transport' > 'Email Address Policies'. Here you will see the default policy and you can select 'Edit' on the left hand side of the MMC console.

2. We are editing the default policy so just click 'Next'.

3. Again, just select 'Next' because we are editing the default policy for all objects.

4. Now edit the default email address. Because mine is set as an alternate UPN I can click the SMTP address and click 'Edit'.

5. As the email address I want to use is an alternate UPN and can select it from the drop down box and select 'OK'.

6. Click 'Next' to apply changes immediately.

7. Click 'Edit' to make the changes.

8. This dialog shows the Power Shell script along with any warnings. I have an invalid recipient name from my old Exchange 2003 organisation that I must get round to changing! ;-)

9. All done and applied. I like how the 'True' display lets you know that you did apply the policy.