Hybrid - Mail Flow

TJ Singh Partner Technical Consultant

tisingh@Microsoft.com

Om Prakash Nath Partner Technical Consultant

omnath@Microsoft.com
Delivery Schedule
S. No Product Category Date & Time

1 Exchange Online Hybrid Deployment May 21, 2015

2 Exchange Online Mail flow June 4, 2015

3 Exchange Online Public Folders June 18, 2015

4 Exchange Online Outlook Connectivity issues July 2, 2015

5 Exchange Online Identity July 16, 2015

6 SharePoint Online SharePoint Online July 30, 2015

7 One Drive One Drive August 13, 2015

8 Lync Online Lync Online August 27, 2015

 Agenda

Planning And Deployment

Office 365/Exchange Planning and deployment tool

Mail flow Scenarios:
Mail routing and customer type
Connector and mail routing end to end scenarios
Avoid common mistakes

Troubleshooting tools

Office 365 mail flow troubleshooting index

Remote Connectivity Analyzer tool
Mail flow guided walkthrough for office 365

Top Mail Flow Issues

Message Tracking and other troubleshooting tasks

Cannot send email messages
Cannot receive email messages
Slow or delayed email messages
Accepted Domain
SMTP Relay
Planning And Deployment
Office 365/Exchange Planning and deployment tool

https://technet.microsoft.com/en-us/exdeploy2013/Checklist?state=2419-W-AAAAAAAAQAAAAAEAAAAAAAA%7e
 DEMO
Office 365 Deployment Tool
Mail flow Scenarios
Mail routing and customer type
Mail flow Scenarios
Connector and mail routing end to end scenarios

Fully hosted

Protection only MX points to EOP

Hybrid MX points to EOP

Hybrid MX points to on-premise

Hybrid MX points to EOP, CMT enabled

Hybrid MX points to service provider

 4
2

1
 4.1

3.2
1.1
1.2 3.1
2.1

2.2

4.2
 12
11 4.1

9 7.2
7.1 8
3.2
10 5
3.1
2.1 6
1.2 1.1

2.2

4.2
 8

9 5
10.2 7
3.2
4.2
3.1
1.2
2.2
6 4.1
1.1
4.3

2.3

10.1 2.1
 11.1 4.1

9.3 7.1
5
10.1 2.2 3.1 7.2
8.3 3.2
6 1.1
2.1 4.2 8.2
1.2
8.1 9.2
9.1 10.3
10.2 11.3
11.2
 3.2
7.2
8

9 7.1
11.2 2.3 5
4.1
1.2 10.2 3.1
4.3 11.1
6
1.1
4.2
2.2

10.1
9.1
2.1
Avoid common mistakes
1. All EOP and Hybrid customers must have inbound and outbound connector of type OnPremises
2. Test mail flow / configuration using Remote Connectivity Analyzer
3. Do NOT create inbound connector of type OnPremises when using 3rd party service provider. Create
Partner connector
4. Be very careful when using IP restriction in inbound connector, it will reject mail when connection IP
address does not match
5. InternalRelay domain requires outbound connector

6. When using Centralized Mail Transport(a.k.a. CMC)

Must have inbound connector of type OnPremises

7. Make sure smart host in outbound connector is correctly configured

Troubleshooting Mail Flow

The first step in knowing how to troubleshoot mail delivery issues is to understand how mail flows.

The path of mail flow will differ depending on whether the environment is an Office 365 (Cloud Only) user

connection or a coexistence environment between on-premises and Exchange Online.

If necessary, review the mail flow information earlier in this module for each of these environments:

Cloud Only

On-premises Relay and Cloud (coexistence).

Golden Rules of troubleshooting

On Premises to Office 365 Office 365 to On Premises

Get-Queue Message Delivery Report

Get-MessagingTrackingLog Message Tracing Report
Event Viewer Protocol Log (On Hybrid Exchange Server)
Protocol Logging NDR
NDR Mail Header
Mail Header
Mail flow from On Premises to Office 365

Mails are not going from On Premises Exchange server to Office 365

Start with Get-Queue & Get-MessageTrackingLog

These commands will let you know where and why mails are stuck
Get-Queue
[PS] C:\>Get-Queue

Identity DeliveryType Status MessageCount NextHopDomain

-------- ------------ ------ ------------ -------------
EX1\10 DnsConnec... Ready 0 exchangeinside.mail.onmicrosoft.com
EX1\Submission Undefined Ready 0 Submission
EX1\Unreachable Unreachable Ready 1 Unreachable Domain
Detailed Output
[PS] C:\>Get-Queue |fl

RunspaceId : bc37517f-1e7b-40f2-b2b2-a6ee68986ef0

DeliveryType : Unreachable

NextHopDomain : Unreachable Domain

TlsDomain :

NextHopConnector : 00000000-0000-0000-0000-000000000000

Status : Ready

MessageCount :1

LastError :

LastRetryTime :

NextRetryTime :

DeferredMessageCount : 0

QueueIdentity : EX1\Unreachable

Identity : EX1\Unreachable

IsValid : True
Get-MessageTrackingLog
[PS] C:\>Get-MessageTrackingLog -Sender ex14local@cloud365.in -MessageSubject:xxx

EventId Source Sender Recipients MessageSubject

------- ------ ------ ---------- --------------
SUBMIT STORE... ex14local@cloud365.in {} xxxxxxxx
RECEIVE STORE... ex14local@cloud365.in {ex14cloud@cloud365.in} xxxxxxxx
RESOLVE ROUTING ex14local@cloud365.in {ex14cloud@EXCHANGEINSIDE.mail... xxxxxxxx
TRANSFER ROUTING ex14local@cloud365.in {ex14cloud@EXCHANGEINSIDE.mail... xxxxxxxx
Mail flow from Office 365 to On Premises

When mail flow is affected from Office 365 to On Premises

Start with the following steps:

Message Trace in Office 365

Mail delivery report in Office 365

Message Trace in Office 365

Here, the Status field will let us why the mail is not delivered
In case of an issue, the status will be either differed or pending
Delivery report in Office 365

Delivery Report for

EX15Local (EX15Local@cloud365.in)

Transferred

4/9/2014 7:04 PM

The message was transferred to another

part of your organization's e-mail system.
Final delivery couldn't be confirmed at this
time.
Event Log in Local Exchange server

Log Name: Application

Source: MSExchangeTransport

Date: 3/24/2014 12:10:17 PM

Event ID: 2022

Task Category: SmtpSend

Level: Error

Keywords: Classic

User: N/A

Computer: EX1.onprem.local

Description:

Outbound TLS authentication failed with error RevocationOffline for Send

connector Outbound to Office 365. The TLS authentication mechanism is
DomainValidation. Target is exchangeinside.mail.onmicrosoft.com.
Protocol logging
2014-03-24T12:29:32.810Z,EX1\Inbound from Office 365,08D1157114542214,3,10.0.86.146:25,213.199.154.10:30372,<,EHLO emea01-am1-obe.outbound.protection.outlook.com,

2014-03-24T12:29:32.810Z,EX1\Inbound from Office 365,08D1157114542214,4,10.0.86.146:25,213.199.154.10:30372,>,250-cloud365.in Hello [213.199.154.10],

2014-03-24T12:29:32.810Z,EX1\Inbound from Office 365,08D1157114542214,5,10.0.86.146:25,213.199.154.10:30372,>,250-SIZE 10485760,

2014-03-24T12:29:32.810Z,EX1\Inbound from Office 365,08D1157114542214,6,10.0.86.146:25,213.199.154.10:30372,>,250-PIPELINING,

2014-03-24T12:29:32.810Z,EX1\Inbound from Office 365,08D1157114542214,7,10.0.86.146:25,213.199.154.10:30372,>,250-DSN,

2014-03-24T12:29:32.810Z,EX1\Inbound from Office 365,08D1157114542214,8,10.0.86.146:25,213.199.154.10:30372,>,250-ENHANCEDSTATUSCODES,

2014-03-24T12:29:32.810Z,EX1\Inbound from Office 365,08D1157114542214,9,10.0.86.146:25,213.199.154.10:30372,>,250-STARTTLS,

2014-03-24T12:29:32.810Z,EX1\Inbound from Office 365,08D1157114542214,10,10.0.86.146:25,213.199.154.10:30372,>,250-AUTH,

2014-03-24T12:29:32.810Z,EX1\Inbound from Office 365,08D1157114542214,11,10.0.86.146:25,213.199.154.10:30372,>,250-8BITMIME,

2014-03-24T12:29:32.810Z,EX1\Inbound from Office 365,08D1157114542214,12,10.0.86.146:25,213.199.154.10:30372,>,250-BINARYMIME,

2014-03-24T12:29:32.810Z,EX1\Inbound from Office 365,08D1157114542214,13,10.0.86.146:25,213.199.154.10:30372,>,250 CHUNKING,

2014-03-24T12:29:32.966Z,EX1\Inbound from Office 365,08D1157114542214,14,10.0.86.146:25,213.199.154.10:30372,<,STARTTLS,

2014-03-24T12:29:32.966Z,EX1\Inbound from Office 365,08D1157114542214,15,10.0.86.146:25,213.199.154.10:30372,>,220 2.0.0 SMTP server ready,

2014-03-24T12:29:32.966Z,EX1\Inbound from Office 365,08D1157114542214,16,10.0.86.146:25,213.199.154.10:30372,*,,Sending certificate

2014-03-24T12:29:32.966Z,EX1\Inbound from Office 365,08D1157114542214,17,10.0.86.146:25,213.199.154.10:30372,*,"CN=cloud365.in, OU=IT, O=Om Prakash Nath, L=Bangalore, S=Karnataka, C=IN",Certificate subject

2014-03-24T12:29:32.966Z,EX1\Inbound from Office

Protocol logging
2014-03-24T12:29:34.247Z,EX1\Inbound from Office 365,08D1157114542214,46,10.0.86.146:25,213.199.154.10:30372,<,RCPT
TO:<ex14local@cloud365.in>,

2014-03-24T12:29:34.247Z,EX1\Inbound from Office 365,08D1157114542214,47,10.0.86.146:25,213.199.154.10:30372,>,250 2.1.0 Sender

OK,

2014-03-24T12:29:34.247Z,EX1\Inbound from Office 365,08D1157114542214,48,10.0.86.146:25,213.199.154.10:30372,>,250 2.1.5 Recipient

OK,

2014-03-24T12:29:34.685Z,EX1\Inbound from Office 365,08D1157114542214,49,10.0.86.146:25,213.199.154.10:30372,<,BDAT 12997 LAST,

2014-03-24T12:29:35.138Z,EX1\Inbound from Office 365,08D1157114542214,50,10.0.86.146:25,213.199.154.10:30372,*,,Set mail item OORG

to 'cloud365.in' based on 'MAIL FROM:'

2014-03-24T12:29:35.481Z,EX1\Inbound from Office 365,08D1157114542214,51,10.0.86.146:25,213.199.154.10:30372,*,Tarpit for

'0.00:00:01.859' due to 'DelayedAck',Delivered

2014-03-24T12:29:35.481Z,EX1\Inbound from Office 365,08D1157114542214,52,10.0.86.146:25,213.199.154.10:30372,>,250 2.6.0

<c0b5ac2c9039464884e35efd20cf6e95@SINPR04MB268.apcprd04.prod.outlook.com> [InternalId=4] Queued mail for delivery,

2014-03-24T12:29:35.950Z,EX1\Inbound from Office 365,08D1157114542214,53,10.0.86.146:25,213.199.154.10:30372,<,QUIT,

Mail Header
HeaderName HeaderValue

Content-Type multipart/mixed; boundary="_000_1213B5B092F64642A431AB2C63A6B5496CAD86EX1onpremlocal_"

From EX14Local <ex14local@cloud365.in>

Here, the attribute X-MS-
To EX14-Cloud <ex14cloud@cloud365.in> Exchange-Organization-AuthAs
Subject To O365 is of our interest
Thread-Topic To O365

Thread-Index Ac9HWe1MMgg31fOrTLiI/dA0xkxUhw==
If this attribute is set as
Date Mon, 24 Mar 2014 12:09:43 +0000
Internal , it signifies that the
Message-ID <1213B5B092F64642A431AB2C63A6B5496CAD86@EX1.onprem.local>
Mutual TLS in Office 365 is
MIME-Version 1.0
working as expected
Return-Path ex14local@cloud365.in
In case, if you find this value as
X-MS-Exchange-Organization-AuthAs Internal Anonymous, you can safely
X-MS-Exchange-Organization-AuthMechanism 04 assume that the mutual TLS is
X-MS-Exchange-Organization-AuthSource EX1.onprem.local broken and the mail flow is
X-MS-Exchange-Organization-SCL -1 happing over Opportunistic TLS
X-MS-Exchange-Organization-AVStamp-Service 1.0

X-OriginatorOrg cloud365.in
Non delivery report 1.1
In Exchange 2013, NDRs are designed to be easy to read and understand by both end-users and administrators.
Information that is displayed in an NDR is separated into the following two areas:

A user information section

A Diagnostic information for administrators section

Non delivery report 1.2
The following sections provide examples of two ways that NDR messages can be generated:

By the same server

By different servers

a) NDR generated and original message rejected by the same server

Suppose a remote email organization accepts delivery of an email message through an Edge
Transport server, and then rejects that message because of a policy restriction on the
recipient's mailbox.

In this case, the sender is not allowed to send messages to the recipient. Edge Transport
servers do not perform message size validation so the Edge Transport server in this example
accepts the message because it has a valid recipient address and the message does not
violate another content restrictions.

Because the remote email organization accepts the whole message, including the message
contents, the remote email organization is responsible for rejecting the message and for
generating the NDR message to be sent to the sender.
Non delivery report 1.4
b) NDR generated and original message rejected by different servers

In this example, the remote server rejects the message and returns an enhanced status code to the local sending server
because the specified recipient does not exist. The rejection happens before the receiving server ever acknowledges the
message. Because the receiving server doesn't successfully acknowledge the message, the receiving server is not
responsible for the message. Therefore, the local sending server generates the NDR message and sends it to the sender of
the original message.
Non delivery report 1.3
NDR generated and original message rejected by the same server
Non delivery report 1.5
NDR generated and original message rejected by different servers
 Receive Connectors [Important Parameters]
[PS] C:\>Get-ReceiveConnector *office* | fl *tls*
SuppressXAnonymousTls : False
RequireTLS : True
TlsDomainCapabilities : {outlook.com:AcceptOorgProtocol}

PS D:\> Get-InboundConnector *hybrid* |fl *tls*

RequireTls : True
TlsSenderCertificateName : cloud365.in
 Send Connectors [Important Parameters]
PS D:\> Get-OutboundConnector *hybrid* |fl *tls*

TlsDomain : cloud365.in
TlsSettings : DomainValidation

[PS] C:\>Get-SendConnector *office* |fl *tls*

TlsDomain : outlook.com
TlsAuthLevel : DomainValidation
IgnoreSTARTTLS : False
RequireTLS : True
Delivery Schedule
S. No Product Category Date & Time

1 Exchange Online Hybrid Deployment May 21, 2015

2 Exchange Online Mail flow June 4, 2015

3 Exchange Online Public Folders June 18, 2015

4 Exchange Online Outlook Connectivity issues July 2, 2015

5 Exchange Online Identity July 16, 2015

6 SharePoint Online SharePoint Online July 30, 2015

7 One Drive One Drive August 13, 2015

8 Lync Online Lync Online August 27, 2015