Saturday, September 26, 2015

Can't Create Domain Certificate with Custom Template in IIS Manager

I was teaching an informal class this week on certificates and Active Directory Certificate Services (AD CS). When a class is informal, you don't necessarily create detailed lab activities ahead of time. It's more like you have a general flow you want to do. Consequently, something just don't work as you expect.

We used IIS Manager to create a domain certificate from the internal CA that we has setup. That part worked fine based on the Web Server template that exists automatically after installing AD CS.

After that I thought this would be a great way to demonstrate superceding a template. Turns out I was wrong. The wizard in IIS Manager will only request certificates based on the Web Server template. If you take that one away, the wizard for creating a domain certificate doesn't work. We were using Windows Server 2012 R2, but this behavior is also consistent in Windows 2008/2008 R2 (IIS 7/7.5)

The work around is to use the Certificates snap-in to create the certificate from AD CS.


Sunday, September 20, 2015

iOS 9 Increases Security Requirements

Apple recently released iOS 9 and this is important for Exchange administrators. Many implementations of Exchange 2010 are implemented on Windows Server 2008 R2 which does not meet the security requirements for iOS 9 to connect.

Apps on iOS 9 or later and OS X 10.11 or later use App Transport Security. App Transport Security defines minimum security standards that must be met for a network connection to be successful. The default behavior requires:
  • TLS version 1.2
  • Perfect forward secrecy
  • SHA256 or better for certificate signature
  • 2048 bit or better RSA key (or 256 bit or better ECC key)

If your Exchange server does not meet the App Transport Security requirements then the connection is dropped by iOS or OS X. This applies to web services. So, it will affect OWA, ActiveSync, and EWS (Outlook for Mac).

We are just starting to see iOS 9 installed by clients and are in the process of ensuring that servers meet the requirements.

NOTE: As of right now (Sept 20/15), iOS 9 does not appear to be enforcing the above requirements for internal applications but does for third party applications. We are continuing to update all Exchange servers to meet these requirements as they are best practices for security and because they could affect OS X 10.11 when released.

TLS 1.2

TLS 1.2 has been around since 2008 but is not enabled by default on Windows Server 2008 R2. Windows Server 2012 and later have TLS 1.2 enabled by default. Windows Server 2008 does not include TLS 1.1 or TLS 1.2.

To enable TLS 1.2 on Windows Server 2008 R2, you need to create a registry key and reboot the server:
  1. Open Regedit.
  2. Browse to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols.
  3. In Protocols, create a new Key named TLS 1.2.
  4. In TLS 1.2, create a new Key named Server.
  5. In Server, create a new DWORD (32-bit) Value:
    • Name: DisabledByDefault
    • Value: 0
  6. In Server, create a new DWORD (32-bit) Value:
    • Name: Enabled
    • Value: 1
  7. Reboot the server.


Perfect Forward Secrecy

Perfect Forward Secrecy ensures that each connections uses a difference encryption key. Windows Server 2008 R2 support the use Perfect Forward Secrecy, but again, it's not enabled by default. There are a limited number of ciphers that use Perfect Forward Secrecy. It is a matter of enabling these ciphers and also changing the order of preference so that they are preferred.

I've done quite a bit of searching and there is no good guide for manually enabling Perfect Forward Secrecy. However, there are several scripts you can download and even a graphical tool.

My personal favorite is the graphical tool IIS Crypto from Nartac Software because it provides an easy way to see exactly which protocols and ciphers are being modified. Press the Best Practices button to view the optimal security settings and then Apply and reboot the server. This will enable TLS 1.2, disable SSL 2 and SSL 3 and enable Perfect Forward Secrecy in one step.

Script to enable Perfect Forward Secrecy and more:


SHA is the algorithm used to sign the digital certificates installed on your Exchange server. Many older certificates use SHA1 which is relatively insecure. You can update your certificate by regenerating the certificate. Most provides allow you to regenerate the keys at no cost. During this process you can select to use SHA256 and then import in Exchange Server.

2048 Bit Key

Several years ago, certification authorities started requiring a minimum key length of 2048 bits. So, it is unlikely that this is an issue for your certificate. However, if you had a certificate with a long lifetime, you can regenerate it with a 2048 bit key to bring it up to requirements.

Testing Your Configuration

The simplest way to verify the SSL/TLS configuration on your server is to use the SSL test from Qualys SSL Labs. This web site connects to your server and runs it through a variety of tests before providing a report on the SSL configuration. It's the fastest way to find out whether specific versions of SSL/TLS and ciphers are enabled or disabled.

Windows Server 2012 and Windows Server 2012 R2

This blog post has focused on Windows Server 2008 R2 because there is a lot of it out there running Exchange 2010. However, Windows Server 2012 and Windows Server 2012 R2 are used for many implementations of Exchange 2013.

Windows Server 2012 and Windows Server 2012 R2 support TLS 1.2 by default. If you have installed update KB2919355 released in April 2014, then Windows Server 2012 R2 uses Perfect Forward Secrecy. For Windows Server 2012, you need to configure the cipher protocols to enable Perfect Forward Secrecy just as was required in Windows Server 2008 R2. Again, using a script or IIS Crypto is the preferred method of enabling and reordering the necessary ciphers.


Links for reference:

Saturday, September 12, 2015

Cannot achieve Exchange Server authentication

In a single server Exchange environment, you can get away with some misconfiguration because the Exchange servers don't need to communicate with each other. Recently I was working with a larger client that made a common configuration error on receive connectors. This client had the following configuration:
  • multi-role DAG members in datacenter #1
  • multi-role DAG members in datacenter #2
  • all mailbox databases are replicated between the two datacenters
  • 1 public folder database in each datacenter
Message delivery was working properly between servers in datacenter #1. Only passive database copies were in datacenter #2. However, there were issues with public folder replication. Some public folders were not replicating properly from datacenter #1 to datacenter #2.

Looking at the queues for servers in datacenter #2, you could see that the queue between AD sites had a large number of messages. The following error was present on the queue:
451 4.4.0 Primary target IP address responded with: "451 5.7.3 Cannot achieve Exchange Server authentication"
The servers had been configured with a receive connector for relaying with only anonymous authentication. This had been done to allow various applications to relay messages through as necessary. On the Network tab of the relay connector, Receive mail from remote servers that have the IP addresses was configured with a range of IP addresses that included the Exchange servers.

When an Exchange server receives an inbound request for SMTP, it uses the source IP address of the request to select the appropriate receive connector. The preferred receive connector has the most specific IP address range that includes the source IP address. A single IP address is preferred over a range and small range is preferred over a larger range. If connectivity fails due to authentication errors no other connectors are used and communication fails.

Normally when Exchange servers send SMTP messages between themselves, they use the Default receive connector which is configured for Exchange Server authentication. The Default receive connector has a source IP range of When the server is first installed, this ensures that communication works properly. However, any receive connector you create is more specific than this and will be preferred over the Default receive connector.

That is what happened in this case. The relay connector that included the IP addresses of the Exchange servers was preferred over the Default receive connector that had the proper authentication settings. This is a common error.

The fix was to add the specific IP addresses of the Exchange servers to the Default receive connector on each server. This made the Default receive connector more specific than the relay connector.Once this was done the error disappeared and queues delivered messages properly.

One question is, why was only public folder replication affected and not mail flow within the AD site?

Within an AD site, the Hub Transport role on a server communicates directly with the Mailbox role on another server with the mailbox database by using RPC. So, the Default receive connector is not required for message delivery within the AD site.

In this case, the only inter-site SMTP messages were the public folder replication messages. No databases were mounted in Datacenter #2 . So, no email messages were sent there via SMTP. Database replication does not use SMTP.

The mail flow configuration in this scenario should be relevant for Exchange 2007, Exchange 2010, and Exchange 2013. However, the error message may vary slightly.

Tuesday, September 8, 2015

Hung at Couldn't Complete Updates Undoing Changes

Setting up a new VM for a client today and ran into a new issue. Well new to me. Fortunately others have seen this before.

In a brand new Windows Server 2012 R2 VM (provided by Dell), I downloaded all of the available updates from Windows Update and started the installation. After a long period of time, the VM rebooted and I got the following message:
We couldn't complete the updates
Undoing changes
Don't turn off your computer
I'm a reasonably patient guy, but after 40 minutes of this I decided that waiting longer wasn't going to fix this. The first round of searching suggested that I disable Secure Boot for the VM. Apparently others had good luck with this:
Unfortunately I restarted the VM and Windows Server 2012 R2 hung in exactly the same spot again. Although I only waited about 10 minutes this time before declaring it hung. So, what's next.

There is at least one Windows Update that expects Bitlocker to be installed for Gen 2 VMs. You don't need to encrypt anything but the feature needs to be installed. On my system it was Security Update KB2920189 that seemed to cause the issue.

You might be able to install BitLocker by booting the VM into safemode. However, there is a better way. On the Hyper-V host, you can use Server Manager to add the feature to the VM while it is turned off.

To see the details with beautiful screenshots, check out this link:
After installing the BitLocker feature in the VM, some updated completed installing quickly and I could log in. Most of the updates failed and I had to run Windows Update again, but all was good.

Sunday, August 23, 2015

Framed Route Configuration for MTS Internet

I recently found out that you can have multiple IPs on a MTS (Manitoba Telecom Services) Internet connection. I haven't investigated this for a long time and this was a pleasant surprise when migrating a client from a more expensive Internet connection to MTS after MTS upgraded the connectivity in their relatively remote building.

Adding additional IPs with MTS is done by purchasing a framed route. The minimum size is 6 IP addresses for $9.95/month. Pretty good if you need an extra IP or two as our client did. The description of the service is here:
We have the framed route up and running for the customer and it worked very well once it was configured. What we ran into was vague documentation and a support person that wasn't familiar with it and messed up the configuration on the first attempt (a second tech was very helpful and got it right).

MTS provides the following document to clients:
The document provided by MTS pretty much just indicates to add an IP from your framed route range to your computer or firewall and it will all start working. However, that is not the case. Your modem from MTS needs to be configured to support the framed route. It's not hard to do, but there is no documentation available, and it's not widely understood by the support staff.

We did the install for a Pace modem. So, the screenshots below show what needs to be configured for that specific modem type.

After you have connected to your Pace modem, go to Settings > Broadband > Link Configuration.

At the bottom of this page, in the Supplementary Network section, select Enable to add an additional network.

Notice that the interface doesn't ask you for your network, it only asks for the address you want to use as the default gateway (router address) and the subnet mask. The Pace box identifies the network from this information. So, you need to determine ahead of time which IP address in your framed route that you want to use as gateway. This means if you purchased a framed route of 6 IP addresses, only 5 are usable because the gateway uses one IP address.

You should also select the Auto Firewall Open option to automatically forward all Internet traffic to hosts inside the Pace box. If you don't, you'll need to either forward specific ports in the Pace firewall, or configure additional DMZ zones in the Pace box. In our case, we were configuring a firewall behind the Pace box and wanted all traffic to be forwarded.

Tuesday, August 4, 2015

Optimize Network Connectivity for Office 365

Microsoft has just released a nice video on optimizing network connectivity for Office 365:
A quick summary of the video:

A lot of connectivity to Office 365 is not in your control. However, you should have an understanding of connectivity to the data center to your tenant. In some cases, MS has worked with ISPs to optimize connectivity in cases where there was obviously bad routing. For example traffic being routed unnecessarily over trans-oceanic links.

If you have high latency to Office 365 you can use tools such as PSping to look at connectivity. PSping performs a connection to a service at a port number rather than using ICMP packets as regular ping does. You can also use Network Monitor to identify some issues.

PSping is free from Sysinternals:
It's important to know that DNS lookups are based on geographic location.  For example will resolve to a different data center depending on location. If users are accessing through a corporate VPN they may be accessing inefficiently.

Wednesday, July 15, 2015

Creating Shared Mailbox in a Hybrid Deployment

The first thing to be aware of when creating shared mailboxes in a hybrid deployment is security. Sharing mailboxes between on-premises and O365 is not supported. So, if a group of people need to share a mailbox then their mailboxes all need to be on-premises or all in O365.

On-Premises Shared Mailboxes

Creating an on-premises shared mailbox is pretty straight forward. Create the shared mailbox in the on-premises Exchange and it all works.

In Exchange 2013, shared mailboxes are explicitly listed as a recipient type in the Exchange admin center (EAC). You can create and manage the shared mailboxes there.

In Exchange 2010, shared mailboxes are not part of the Exchange Management Console (EMC). You need to create the shared mailbox by using the New-Mailbox cmdlet in the Exchange Management Shell (EMS). For example:
New-Mailbox HelpDesk -shared -UserPrincipalName
After creating the shared mailbox in Exchange 2010, you need to give users permission to access it. Assign Full Mailbox permissions to let users manage the contents of the mailbox. You may also want to give SendAs permissions depending on your scenario.

Office 365 Share Mailboxes

In Office 365, the web-based management interface provides the same option to create shared mailboxes as Exchange 2013 does. However, in a hybrid environment, you can't create the shared mailboxes directly in Office 365.

If you create the share mailbox directly in Office 365 there is no Active Directory reference to the shared mailbox on-premises. This prevents Outlook from properly adding the shared mailboxes because autodiscover does not work properly. In a hybrid environment, autodiscover is directed to the on-premises Exchange organization and won't be able to direct Outlook to the correct location of the shared mailbox because there is no information in Active Directory about the shared mailbox in the on-premises AD.

In a hybrid environment, you should perform the following steps instead:
  1. Create a Remote Mailbox in Office 365 from the on-premises Exchange organization.
  2. Run Dirsync (or wait for several hours).
  3. In Office 365, convert the mailbox to a shared mailbox. Available when the recipient is selected as seen in the screenshot to the right.
  4. In Office 365, configure Full Access and SendAs permissions to the shared mailbox as required.
It's a bit more of a hassle to create a shared mailbox in Office 365 for a hybrid environment, but it does work!

When you create the shared mailbox directly in Office 365, you'll see the following symptoms:
  • Shared mailboxes are not automatically added to Outlook.
  • If you attempt to add the shared mailbox to Outlook manually in the properties of the Exchange account, then Outlook will continually prompt for credentials and hang.