TFS Planning Guide v1.3 PDF
TFS Planning Guide v1.3 PDF
TFS Planning Guide v1.3 PDF
The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as
of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be
a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the
date of publication.
This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY,
AS TO THE INFORMATION IN THIS DOCUMENT.
Microsoft grants you a license to this document under the terms of the Creative Commons Attribution 3.0 License. All other
rights are reserved.
2013 Microsoft Corporation.
Microsoft, Active Directory, Excel, Internet Explorer, SQL Server, Visual Studio, and Windows are trademarks of the Microsoft
group of companies.
All other trademarks are property of their respective owners.
Page 2 of 109
Planning Guide Foreword
Table of Contents
Foreword ............................................................................................................................................................................................................................................ 5
Introduction ...................................................................................................................................................................................................................................... 6
Whats changed? ............................................................................................................................................................................................................................. 8
TFS 2012 ........................................................................................................................................................................................................................................ 8
TFS 2013 ........................................................................................................................................................................................................................................ 8
Defining your TFS Strategy ......................................................................................................................................................................................................... 9
Understanding TFS Topologies ........................................................................................................................................................................................ 10
Deciding on service versus on-premises server infrastructure ........................................................................................................................... 11
Determining on-premises server infrastructure capacity planning strategy ................................................................................................ 13
Considering Upgrade and Migration ............................................................................................................................................................................. 17
Considering Virtualization .................................................................................................................................................................................................. 19
High Availability Strategies ................................................................................................................................................................................................ 19
Build Server Considerations ............................................................................................................................................................................................... 20
Proxy Server Considerations .............................................................................................................................................................................................. 21
Defining your Team Project Collection Strategy ............................................................................................................................................................ 23
Understanding Team Project Collections .................................................................................................................................................................... 24
Key Decisions Single versus Multiple Team Project Collection Strategy .................................................................................................... 26
Deciding on Team Project Collection Strategy .......................................................................................................................................................... 26
Defining your Team Project Strategy .................................................................................................................................................................................. 29
Understanding Team Projects .......................................................................................................................................................................................... 30
Team Project Boundaries .................................................................................................................................................................................................... 31
Team Project Constraints .................................................................................................................................................................................................... 32
Key Decisions Single versus Multiple Team Project Strategy .......................................................................................................................... 33
Deciding on a Team Project Strategy ............................................................................................................................................................................ 34
Defining your Team Strategy .................................................................................................................................................................................................. 38
Understanding Teams .......................................................................................................................................................................................................... 39
Team Boundaries .................................................................................................................................................................................................................... 39
Team Constraints ................................................................................................................................................................................................................... 39
Key Decisions Single versus Multiple Team Strategy .......................................................................................................................................... 40
Area Paths and Teams.......................................................................................................................................................................................................... 40
Deciding on Team Strategy ............................................................................................................................................................................................... 40
Defining your Disaster Recovery (DR) Strategy .............................................................................................................................................................. 44
Understand DR Strategies .................................................................................................................................................................................................. 44
Understand DR Avoidance ................................................................................................................................................................................................. 46
DR Recovery Walkthroughs ............................................................................................................................................................................................... 51
Large and Complex Project considerations ...................................................................................................................................................................... 53
Page 3 of 109
Planning Guide Foreword
Understanding Large Projects .......................................................................................................................................................................................... 54
Deciding on a Large Project Strategy ............................................................................................................................................................................ 55
Strategy 1 - Single Team Project Collection examples .......................................................................................................................................... 57
Strategy 2 - Multiple Team Project Collections examples .................................................................................................................................... 58
Questions & Answers ................................................................................................................................................................................................................. 62
Real World Reference Stories ................................................................................................................................................................................................. 64
Scenario 1 Size doesnt matter ..................................................................................................................................................................................... 65
Scenario 2 Getting complex ........................................................................................................................................................................................... 66
Scenario 3 New releases every day ............................................................................................................................................................................. 67
Scenario 4 Across the globe .......................................................................................................................................................................................... 68
Scenario 5 No time for refactoring ............................................................................................................................................................................. 69
Scenario 6 Disaster by hotfix - Vladimirs DR Experience .................................................................................................................................. 70
Scenario 7 Aborted Warehouse Processing - Thorstens DR Experience ................................................................................................... 71
Appendix ......................................................................................................................................................................................................................................... 73
Enabling and using TFS Logging Examples ................................................................................................................................................................ 73
Working with TFS Performance Counters .................................................................................................................................................................... 77
DR Recovery Walkthroughs ............................................................................................................................................................................................... 80
Authoring Reports ................................................................................................................................................................................................................. 97
Sample Maintenance Scripts ........................................................................................................................................................................................... 100
Quick Reference Sheets .......................................................................................................................................................................................................... 103
Page 4 of 109
Planning Guide Foreword
Foreword
In 2005, within six months of joining the Team Foundation Server (TFS) v1 team, I was given the task to go to
Ferrari and install TFS. The assignment was actually more than a simple installation, involving server capacity
planning and developer training all crammed within 4 days. As I soon found out, designing and implementing a
deployment that can scale as an enterprise organization either increases in size, increases its project complexity
or adds more departments is a challenging task. There are a lot of variables, opinions and at times plain
guesses - I mean estimations - at play. During that trip I still vividly remember working with one of the Ferrari
managers until 1AM creating a spreadsheet with data that was going to assist us in making the right deployment
decisions. We collected data about # users, # of projects, the size of the version control tree and the expected
load of their build system. It was fun!
The product has evolved and matured since those early days. We added a scale-out solution, optimized for very
large source trees, introduced teams and authored a number of ALM training resources. Clearly we have grown,
but so has the need and complexity of capacity and project planning. An updated, thorough and technical
document is needed. Enter the Rangers and the creation of a comprehensive solution that marries the product
teams knowledge with all of the fields experience. The document will provide you with the concepts, tools and
best practices needed to tackle the heterogeneous nature of your customer engagements. Just like in 2005, it
continues to be fun seeing us grow!
Page 5 of 109
Planning Guide Introduction
Introduction
This guidance delivers practical and scenario based guidance for the implementation of Team Foundation Server
(TFS). We guide you through the decisions whether to have one or more Team Foundation Servers, one or more
Team Project Collections, one or more Team Projects and one or more Teams, based on scenarios and
implications of each decision. We conclude with disaster recovery planning, frequently asked questions and a
collection of real world reference stories.
This version (v1.3) of the guidance is focused on Team Foundation Server 2012 and 2013, whereby most of the
NOTE
concepts and guidance are backwards compatibleoften identicalwith Team Foundation Server 2010.
Contributors
Chris Wishart, Daniel Meixner, Ed Holloway, Francisco Fagas, Gregg Boer, Guy Teverovsky, Jeff Levinson, Jim Szubryt, Joakim
Karlsson, John Berman, Lennart Jansson, Mario Rodriguez, Mike Fourie, Pramod Vasanth, Prasanna Ramkumar, Stefan Mieth,
Thorsten Dralle, Tiago Pascoal, Tim Star, Tina Erwee, Tommy Sundling, Willy-Peter Schaub
1
http://aka.ms/vsarindex
Page 6 of 109
Planning Guide Introduction
How to Use This Guidance
Each section of this guidance has a common content structure:
Topic Overview presents a brief overview of the intent of the
relevant section and mentions the personas that are relevant
(interested) in the section.
Deciding what is Important in this section a quick reference table
that give you are quick map into the topics within the section, based
on your interests.
Understanding the topic delves into the topic and covers topics
such as constraints.
Scenarios presents some common scenarios / strategies that can be
used as reference.
Use the TFS Planning Guidance Hands-on Lab as a practical walkthrough and companion when perusing this
guidance and the Quick Reference Posters and Cheat Sheets for quick reference.
Personas
Refer to Visual Studio ALM Rangers Personas and Scenarios 2 for more information about the personas and
customers profiles referenced in this guide.
2
http://go.microsoft.com/fwlink/?LinkID=230942
Page 7 of 109
Planning Guide Whats changed?
Whats changed?
TFS 2012
This section introduces you to the impact that TFS 2012 has on the guidance around planning TFS environments.
Capacity Planning
With the exception of support for Windows Azure, there are no capacity planning changes compared to TFS
2010.
Refer to Team Foundation Service Practical Guidance 3 for more details and guidance around the Team
Foundation Service scenario.
See Defining your Team Project Collection Strategy on page 23 for details.
Team Projects
No changes with Team Project planning compared to TFS 2010.
Teams
The concept of Teams is new and introduces new planning guidance.
TFS 2013
TFS 2013 has not introduced any changes in terms of the server, team project collection, team project, teams, or
disaster recovery planning. This guide has been reviewed for compliance and minor updates have been added
for SharePoint 2013. Otherwise the concepts and guidance covered herein are backwards compatibleoften
identicalwith Team Foundation Server 2010 and 2012.
3
http://go.microsoft.com/fwlink/?LinkID=230945
Page 8 of 109
Planning Guide Defining your TFS Strategy
Refer to Hands-on Lab, section Stepping through the planning of a Server Strategy, for a walkthrough of this
section.
This section is focused on Dave the TFS administrator and Jane the infrastructure specialist.
Plan the deployment of Application tier (AT) servers when scaling out 14
Page 9 of 109
Planning Guide Defining your TFS Strategy
Single Server
The single-server deployment option places the application and the Data tier on a single server,
whereby build and proxy servers are optional features and can be deployed on the same or
separate servers.
Page 10 of 109
Planning Guide Defining your TFS Strategy
NOTE ALM Rangers dogfooding journal of the Team Foundation Service 4 documents the use of the Team Foundation
Service by the ALM Rangers as part of their 24x7x365 dog-fooding exercise.
4
http://aka.ms/vsarcloudhome
Page 11 of 109
Planning Guide Defining your TFS Strategy
Key Decisions Hosted Service versus On-Premises Server
Feasible Limitations Not feasible
Service Server
TFS Work Items, Source Control and
Build features
Agile Projects Management
Test Case Management
Heterogeneous Development
Near-zero setup and
administration Administration of the operational infrastructure
is a huge cost when using on-premises servers,
even if initiatives such as the VM Factory can
introduce near-zero setup.
Collaborate from anywhere
On-premises infrastructure may introduce
premise-specific authentication and accessibility
constraints.
Virtual Test Lab Management
Not available at the time of
writing this guidance.
SharePoint Integration
Not available at the time of
writing this guidance.
Data Warehouse & Reporting
Not available at the time of
writing this guidance.
Environmental Lack of infrastructure to host TFS
On-premises infrastructure with
unpredictable usage spikes
On-premises infrastructure must be engineered
for extra scalability and fault tolerance to
accommodate the unpredictable spikes, which
is a costly investment that may not meet ROI
criteria.
Anticipated usage that exceeds
on-premises infrastructure On-premises infrastructure must be engineered
capacity to allow scalability without downtime, which is a
costly investment that may not meet ROI
criteria.
Need for total control of
infrastructure for auditing
Multiple partner companies
needing an integrated solution On-premises infrastructure may introduce
premise-specific authentication and accessibility
constraints.
Data in motion / at rest
constraints for compliance
purposes (HIPPA, PCI, etc.), such
as the strictly regulated gaming
industry
Customer Types Humongous Insurance
Trey Research
Consolidated Messenger
Table 2 - Team Foundation Service key decisions and viability
Page 12 of 109
Planning Guide Defining your TFS Strategy
We are not including the basic configuration of TFS in this guidance. It could be considered for the single server
NOTE
deployment scenario, if and only if the integration with SharePoint products or the reporting features are not
required.
The section Real-World Reference Stories on page 5 summarizes a few of the real-world scenarios we have
received, which demonstrate that the minimum hardware specifications might not suffice in the real-world,
especially if performance and scalability is a prime factor.
5
viable feasible not viable
Page 13 of 109
Planning Guide Defining your TFS Strategy
NOTE This guidance includes a planning workbook, TFS Project Guidance - Capacity Planning which you can use to get
feedback about the recommended deployment options and hardware specifications, based on current and
anticipated number of users.
The Real-World Beef Factor is a percentage used to add more resources to the recommended minimum
specifications when working with the companion capacity planning workbook. We recommend that you
experiment with the minimum specifications in your test environment(s) to determine the appropriate Real-
World (Beef) Factor for your environment.
Figure 6 Calculating the number of Application tier (AT) servers in a scale-out scenario
The workbook is based on the following two tables, which summarize the configuration and the request-per-
second (RPS) guidelines:
Configuration RPS Conservative Users
Low End Configuration (Single Server) 92 250
Standard Configuration (Single Server) 180 500
Scale Unit Configuration (Dual Server) 476 2200
High End Configuration (Dual Server) 730 3600
Table 3 - Server configuration styles
Page 14 of 109
Planning Guide Defining your TFS Strategy
Team Size RPS per User Needed (RPS)
250 0.175 44
500 0.175 88
3500 0.225 788
4500 0.2625 1181
Table 3 Request per second (rps) per AT server
This guidance and the capacity planning workbook are intended to be used to calculate recommendations. It is very
NOTE
Scale-up Servers 3 1 x AT: 1 Dual Processor, 4GB RAM, 500GB Disk 2200
4 1 x DT: 1 Quad Core Processor, 8GB RAM, 2TB Disk
Scale-out Servers 7 n x AT: 1 Dual Processor, 4GB RAM, 500GB Disk 3600+
m x DT: 1 Quad Core Processor, 8GB RAM, 2TB Disk
Table 3 Maximum recommended users
Build Servers and Lab Management environments are outside the scope of this guidance. Refer to Build
NOTE
Customization Guidance 6 and Lab Management Guidance 7 respectively for guidance about Build Servers and
Lab Management.
6
http://go.microsoft.com/fwlink/?LinkID=230938
7
http://go.microsoft.com/fwlink/?LinkID=230951
Page 15 of 109
Planning Guide Defining your TFS Strategy
Deployment Option REF8 Example specification Physical Hardware
Max users
Scale-up Servers 3 1 x AT: 1 Dual Processor, 4GB RAM, 500GB Disk 2200
4 1 x DT: 1 Quad Core Processor, 8GB RAM, 2TB Disk
Scale-out Servers 7 n x AT: 1 Dual Processor, 4GB RAM, 500GB Disk 3600+9
m x DT: 1 Quad Core Processor, 8GB RAM, 2TB Disk
Table 4 Recommended maximum users for on-premises server deployments
If you use SharePoint 2010, the installation of TFS will enforce a minimum of 4GB RAM and raise a warning if the
NOTE
system has less than 10GB RAM. This is a SharePoint, not a TFS constraint.
If you use SharePoint 2013, the SharePoint 2013 Distributor Cache which is a new feature which reserves 10% of
RAM on any machine it get started at minimum . Its not recommended to install Distributor Cache on a Server
Running SQL. Read more here 10.
8
REF Reference numbers used by the Cheat Sheets when referring to these hardware specifications
9
Please note that there is a maximum number of users, calculated for AT and DT servers combined. You can scale up to 10,000 or more users as long as they are distributed across different
AT and DT servers.
10
http://blogs.technet.com/b/uktechnet/archive/2013/05/07/guest-post-distributed-cache-service-in-sharepoint-2013.aspx
11
http://msdn.microsoft.com/en-us/library/ms245478.aspx
Page 16 of 109
Planning Guide Defining your TFS Strategy
Planning based on number of Team Project Collections Option 3
The soft limit imposed by the number of active Team Project Collections is summarized in Table 5, whereby TFS
can support from 10 (low end) to 8000 (high end) Team Project Collections per SQL Server instance:
Deployment REF Example specification Max Active 12 Team Project
Option Collections13
Scale-out Servers 7 n x AT: 1 Dual Processor, 4GB RAM, 500GB Disk 75+
m x DT: 1 Quad Core Processor, 8GB RAM, 2TB Disk
Table 5 Recommended max active TPCs for on-premises server deployments
Refer to Defining your Team Project Collection Strategy on page 23 for more information about Team Project
Collections.
12
An active collection is one that will be accessed on a daily frequency.
13
Per SQL Server Instance
14
Rough estimate, which is dependent on modifications made to the process template.
Page 17 of 109
Planning Guide Defining your TFS Strategy
The number of processors that you can make available will help for a good portion of the upgrade
process. The import is multithreaded with the exception of a few steps, so making at least eight
processors available will be beneficial.
Follow SQL Server best practices of having a TempDB datafile per processor. The upgrade process
makes heavy use of TempDB for index creation and processing with Common Table Expressions (CTEs).
Review the Capacity for tempdb guidance 15
Follow SQL Server best practices of having the data, log and TempDB files on separate
drives/mountpoints.
Consider changing the default value MAXDOP setting in SQL Server if your installation kept the default
of zero. Having a MAXDOP equal to half of the number of processors will benefit the multithreaded
processing. Review the Recommendations and Guidelines for max degree of parallelism configuration
16
guidance.
Delete workspaces that have not been used for a while. As a practice, Microsoft notifies workspace
owners of a pending deletion of their workspace if it hasnt been used in the last six months. Builds
create workspaces and you might be surprised to find how much space you could reclaim in your
database by deleting unused workspaces.
If you are not defragmenting your TFS database(s) and re-indexing them you should do this before
you perform a migration. Optimized indexes will help when the migration is processing.
Pin SQL Server memory to use a minimum of 65% and a maximum of 80%. You will see that the
upgrade process will take the maximum amount of SQL memory at points during the processing.
If you have a multi AT environment, remember that the upgrade process can run off of any of the TFS AT
servers. In the case of running the import command on one server, it doesnt mean that steps will run
exclusively from that server. You will want the servers in the multi AT environment to have the same
resource allocation.
You should have your warehouse processing stopped during the migration to conserve your SQL
Server resources and reduce database contention.
Plan for several trial runs if your production development outage window is critical to your business. Your
testing environments resources should match your production environment where you will be performing your
upgrade. You dont want any unpleasant surprises when performing your upgrade.
Check the recovery mode of your databases because the upgrade process will be generating a lot of
transactions. You want to make sure that you dont run into problems with autogrows or transaction logs
becoming full. The TFS data warehouse can be in simple recovery mode as well as the Reporting Services
databases. By default, these are both in full recovery mode. If you are upgrading TFS 2008, you will want to check
the transaction log file autogrow settings for the TFSVersionControl database. Because this database ends up
being the upgraded TFS 2012 database, it is a good practice to at least double the size of the log file.
You should consider doubling or quadrupling the resources allocated to your TFS environment for the upgrade
NOTE
processing. You can read more about common upgrade and migration scenarios in the Visual Studio ALM Rangers
Upgrade Guidance 17.
Most environments are virtualized and adding/removing resources are relatively easy to do. You should consider
doubling or quadrupling the resources allocated to your TFS environment for the upgrade processing. Once your
TFS environment is live you can scale back on resources in a virtual environment to align more with the guidance
identified earlier in this document.
15
http://msdn.microsoft.com/en-us/library/ms345368.aspx
16
http://support.microsoft.com/kb/2023536
17
http://go.microsoft.com/fwlink/?LinkID=230948
Page 18 of 109
Planning Guide Defining your TFS Strategy
Considering Virtualization
You can virtualize most, if not all, of the TFS infrastructure. Refer to Visual Studio ALM Rangers VM Factory
Tooling and Guidance 18 for guidance on virtualization, tooling and reference solutions.
It is imperative that you validate and dog food your virtualized infrastructure before committing it to your
production environment.
Scale-up Servers 1 x AT: 1 Dual Processor, 4GB RAM, 500GB Disk 2200 1760
1 x DT: 1 Quad Core Processor, 8GB RAM, 2TB Disk
1 x AT: 1 Quad Core Processor , 8GB RAM, 500GB Disk 3600 2880
1 x DT: 2 Quad Core Processor, 16GB RAM, 3TB Disk
Scale-out Servers n x AT: 1 Dual Processor, 4GB RAM, 500GB Disk 3600+ 2880+
m x DT: 1 Quad Core Processor, 8GB RAM, 2TB Disk
Table 7 Recommended maximum users in virtualized environments
18
http://go.microsoft.com/fwlink/?LinkID=230953
19
http://www.microsoft.com/en-gb/download/details.aspx?id=29035
Page 19 of 109
Planning Guide Defining your TFS Strategy
20
http://msdn.microsoft.com/en-us/library/dd793166.aspx
21
http://msdn.microsoft.com/en-us/library/dd793166.aspx#build_system_examples
22
http://msdn.microsoft.com/en-us/library/dd578619.aspx
23
http://blogs.msdn.com/b/willbar/archive/2012/01/08/dogfooding-team-foundation-build-by-the-numbers-december-2011.aspx
Page 20 of 109
Planning Guide Defining your TFS Strategy
This post 24 does a great job of explaining all of the communication that goes on between the Visual Studio
workstation, TFS Proxy and Team Foundation Server. The TFS Proxy does not completely eliminate all traffic from
the workstation to the Team Foundation Server. Authentication of the remote user is still performed against the
Team Foundation Server so there is some communication going back and forth.
The default port of the TFS Proxy is 8081. However, it can be changed and if SSL is needed it can be done over
port 443. Metrics to calculate the need for a Proxy Server is beyond the scope of this guide, but the math is
pretty simple. Generally, if you have a remote team of more than 3-5 people, you should see improvements
when you place a proxy server at their location. There should, however, be a significant geographic distance, or
slow network connectivity between the Team Foundation Server and the TFS Proxy to see the value of
implementing it.
Dont place the TFS Proxy on the Team Foundation Server. This is not the correct configuration for implementing the
NOTE
proxy server.
Installing the TFS Proxy requires the TFS installation media. Connecting the TFS Proxy requires connectivity to the
Team Foundation Server. To use TFS from the Visual Studio client, you have to set the TFS Proxy name and port
in Tools, Source Control, Visual Studio Team Foundation Server.
24
http://blogs.msdn.com/b/tsyang/archive/2008/03/24/how-team-foundation-server-proxy-2008-works.aspx
Page 21 of 109
Planning Guide Defining your TFS Strategy
NOTE Do not specify http:// or port when configuring the Proxy server values in Visual Studio.
The value of the Proxy Server can be found in the TFS Admin Console. Use the URL field on the TFS Proxy page.
Remember that the TFS Proxy is intended to improve the download experience. It will not make the check-in
(upload) process any faster. To verify that developers are using the TFS Proxy you can check the IIS logs on the
server. The logs include relevant information such as the IP address of the workstation, duration to download,
username used to connect to the server and the version of Visual Studio that was used. Applications like
LogParser 25 are beneficial for rolling up statistics of the IIS log.
REFERENCES
Visual Studio Team Foundation Service Guidance 26
Visual Studio ALM Rangers Quick Reference Guidance 27
Visual Studio ALM Rangers VM Factory Tooling and Guidance 28
Visual Studio ALM Rangers Upgrade Guidance 29
Team Foundation Server 2012 Installation Guide 30
Team Foundation Server 2012 Administration Guide 31
Team Foundation Server Architecture 32
Team Foundation Server 2010 Installation Guide 33
Team Foundation Server 2010 Administration Guide 34
Professional Team Foundation Server 35 book
Professional Application Lifecycle Management with Visual Studio 2010 36 book
Team Development using Microsoft Visual Studio Team Foundation Server P&P 37 book
25
http://support.microsoft.com/kb/910447
26
http://go.microsoft.com/fwlink/?LinkID=230945
27
http://vs2010quickref.codeplex.com/
28
http://go.microsoft.com/fwlink/?LinkID=230953
29
http://go.microsoft.com/fwlink/?LinkID=230948
30
http://go.microsoft.com/fwlink/?LinkID=231279
31
http://go.microsoft.com/fwlink/?LinkID=231280
32
http://msdn.microsoft.com/en-us/library/ms252473.aspx
33
http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=24337
34
http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=8175
35
http://www.amazon.com/Professional-Team-Foundation-Server-Programmer/dp/0470943327
36
http://www.amazon.com/Professional-Application-Lifecycle-Management-Programmer/dp/0470484268
37
http://www.amazon.com/gp/product/0735625719
Page 22 of 109
Planning Guide Defining your Team Project Collection Strategy
Refer to Hands-on Lab, section Stepping through the planning of a Team Project Collection Strategy, for a
walkthrough of this section.
This section is focused on Dave the TFS administrator and Garry the lead for development teams.
Page 23 of 109
Planning Guide Defining your Team Project Collection Strategy
Isolation Considerations
The acts of isolating Team Projects in Team Project Collections are concerned with:
Scalability
Backup and Restore
Recovery
Security isolation
Scalability
TFS supports the ability to distribute Team Project Collection databases across multiple SQL Server Instances,
providing an effective scale out solution for your deployment.
example, if you have a requirement to have individual backup and restorable units for a Team Project the only
scenario to accomplish this is to have one Team Project per collection.
For more information and an example scenario for archiving, please refer to Visual Studio TFS Team Project and
Collection Guidance 39.
For more information about splitting and merging Team Project Collections, please refer to Visual Studio ALM
Rangers Team Foundation Server Upgrade Guidance 40.
38
http://msdn.microsoft.com/en-us/library/dd236915.aspx
39
http://msdn.microsoft.com/en-us/magazine/gg983486.aspx
40
http://go.microsoft.com/fwlink/?LinkID=230948
Page 24 of 109
Planning Guide Defining your Team Project Collection Strategy
Security Isolation
As shown in the above illustration, sharing of Team Project artifacts among Team Projects is possible within the
same Team Project Collection. There is, however, an isolation boundary between Team Project Collections and
artifacts cannot be shared across Team Project Collection boundaries. If Team Projects must share artifacts, they
must be contained with the same Team Project Collection.
Page 25 of 109
Planning Guide Defining your Team Project Collection Strategy
41
Refers to ability to categorize Team Projects, which are currently represented in a flat and sorted list of Team Projects within a Team Project Collection.
Page 26 of 109
Planning Guide Defining your Team Project Collection Strategy
NOTE Consider giving the Team Project Collection a descriptive name, as renaming is not supported and the default name
of DefaultCollection might not refer to the default Team Project Collection in future.
As shown inFigure 14, the single Team Project Collection scenario implements one Team Project Collection,
which is similar to a SharePoint Site Collection in terms of isolation and basic unit of recovery.
The advantages you have to consider are:
Simplicity for users
o No need to switch context between Team Project Collections
o No need to search for Team Projects in different Team Project Collections
42
http://go.microsoft.com/fwlink/?LinkID=230948
Page 27 of 109
Planning Guide Defining your Team Project Collection Strategy
Ability to share artifacts between Team Projects
Single database for all Team Project data
The disadvantages you have to consider are:
Security complexity and administration needed to achieve isolation within a Team Project Collection
As the number of active Team Projects grows, the flat list within a Team Project Collection could make
navigation between Team Projects difficult.
43
http://vs2010quickref.codeplex.com/
44
http://go.microsoft.com/fwlink/?LinkID=230948
45
http://msdn.microsoft.com/en-us/magazine/gg983486.aspx
46
http://msdn.microsoft.com/en-us/library/dd236915.aspx
Page 28 of 109
Planning Guide Defining your Team Project Strategy
Refer to Hands-on Lab, section Stepping through the planning of a Team Project Strategy for a walkthrough
of this section.
This section is focused on Dave the TFS administrator, and Garry the lead for development teams.
Page 29 of 109
Planning Guide Defining your Team Project Strategy
to a single Team Project, but still distributed over several databases, for example, Documents in SharePoint Portal or
reports. For the sake of simplicity we skip these exceptions in this topic.
The following data is being defined by a Team Project and therefore logically isolated from other Team Projects.
Process
Every Team Project is based on a process template. Depending on the process template that you use, certain
data, which defines the Team Project process infrastructure, is created and instantiated within a Team Project,
this includes:
o Work Item Categories
o Work Item Types
o Link Types
o Queries
o Areas
o Iterations
o Microsoft Project Mapping Settings
o Groups & Permissions
o Lab Templates
o Build Templates
o Source-Control Settings
o Portal Settings
o Reports
Page 30 of 109
Planning Guide Defining your Team Project Strategy
NOTE Later changes of the base process template will not affect existing instances of Team Projects, but every aspect
defined by a process template can be changed during the lifecycle of a Team Project. See TFS Process Template
Customization Guidance 47 for details. This is also true for the other direction: Changes made to an instance of a
Team Project never effect on the base process template.
Members
Within a Team Project, you define team members to allow them to work on the Team Project with different
permissions. For example, you can create new groups on Team Project level.
Work Items
Whenever you create a work item to track work, requirements or any kind of activity, you assign the work item to
a single Team Project. Once assigned, it can never be moved to another Team Project.
Version Control
When you are working with one of the default process templates, several initial version control items will exist
when you create a Team Project. The source code folder allows you to do all the source code activities that you
are used to, like branching and merging, check-in and check-out, etc. A source code folder always belongs to a
single Team Project.
Build Definitions
Build Definitions are a set of parameters which are used in combination with a certain build process template.
Build Definitions are defined on a Team Project Level.
SharePoint Portal
Optional you can specify a SharePoint portal for a Team Project. This allows you to display data stored in
SharePoint in a Team Project node in Team Explorer. Typically, a SharePoint Portal relates to a single Team
Project.
Besides these rather physical aspects of a Team Project, a Team Project also defines logical settings. These
include permissions for certain actions within the scope of the Team Projects, permissions to view or edit certain
data or permissions for manipulation of permissions.
47
http://vsartfsptguide.codeplex.com/
Page 31 of 109
Planning Guide Defining your Team Project Strategy
Page 32 of 109
Planning Guide Defining your Team Project Strategy
Page 33 of 109
Planning Guide Defining your Team Project Strategy
Projects 48 blog post, which summarizes how we designed the ALM Rangers Team Project strategy and how we would
like to change it unfortunately these designs are the immutable ones the teams have started working with.
Humongous Insurance might decide for the Multiple Team Project strategy, if they are developing several
independent solutions.
Consolidated Messenger might decide for the Multiple Team Project strategy if they have several
development groups developing solutions for several customers.
48
http://blogs.msdn.com/b/willy-peter_schaub/archive/2011/11/18/requirements-management-for-ranger-projects-what-i-wish-we-did-differently-with-our-team-projects.aspx
Page 34 of 109
Planning Guide Defining your Team Project Strategy
Single Team Project Strategy
Concepts
Following the Single Team Project Strategy, as shown in Figure 22, you store all your data inside a single Team
Project but create a substructure within the Team Project to isolate information inside the Team Project.
Downsides
On the downside, the Single Team Project Strategy might introduce:
Finding other ways to sub-structure the work items of your Team Project
o In the past, usage of the hierarchical Attribute Area Path was very common to structure your
work items, so Area Path could be used for structuring into components.
o Starting with TFS 2012, the Area Path is meant to be first choice to assign work items to a
specific team and you could use another attribute to structure work items into components
instead.
o Challenge: This leads to a higher complexity in queries, reports etc.
Finding other ways to structure your source code.
o Depending on what you are developing, you might need to create additional subdirectories or
branches, which might lead to a deeper hierarchies and longer file paths.
o Challenge: This leads to a higher complexity with branching and merging, which requires a visit
of the Branching and Merging Guidance 49.
Permissions must be set inside of the Team Project (Source code, Work items, Builds, Reports) to
control who sees and who can control the Team Project data.
49
http://go.microsoft.com/fwlink/?LinkID=230936
Page 35 of 109
Planning Guide Defining your Team Project Strategy
Multiple Team Projects Strategy
Concepts
When you follow the Multiple Team Projects strategy, as shown in Figure 23, you create multiple Team Projects
to store information isolated from each other. To find out about the level of isolation refer to page 31.
Whenever you have multiple teams that are about to follow different processes, there is no other reasonable way
than creating a new Team Project per process. Besides this, working with multiple Team Projects might be a
good choice for you if your company and development team fulfill the following requirements:
You understand the Single Team Project Strategy, see page 35, but dont feel 100% happy with it.
You want a higher level of (logical) isolation.
Indicators
There are some rudimentary indicators which can help you to decide if a multiple project strategy is feasible:
Your products or projects have different release cycles.
Your products or projects have different stakeholders.
Your products or projects have different source code base and technologies.
Your products or projects have different domain relation.
A single Team Project would be too big and introduce unproductivity and discomfort for users.
None of these indicators are hard criteria, but they can be indicators whether you should create a new Team
Project.
Common examples for the usage of multiple Team Projects are:
Consolidated Messenger
As a consulting company create one Team Project per customer, another Team Project for internal tool
development:
Trey Research
As an ISV, create one Team Project per product, another Team Project for base components, another
Team Project for internal tool development:
Page 36 of 109
Planning Guide Defining your Team Project Strategy
Humongous Insurance
Create one Team Project per product per version:
Downsides
The downside of using multiple Team Projects is that you are losing comfort in some aspects:
You have to find the right Team Project, to find and retrieve the information you need.
You have to keep process templates in sync if you still want all your teams to work with the same
process template.
References
Visual Studio ALM Rangers Quick Reference Guidance 50
Visual Studio ALM Rangers Branching and Merging Guidance 51
MSDN Magazine: Visual Studio TFS Team Project and Collection Guidance 52
50
http://vs2010quickref.codeplex.com/
51
http://go.microsoft.com/fwlink/?LinkID=230936
52
http://msdn.microsoft.com/en-us/magazine/gg983486.aspx
53
http://msdn.microsoft.com/en-US/library/ms242894(v=VS.80).aspx
54
http://msdn.microsoft.com/en-us/library/bb668942.asp
55
http://msmvps.com/blogs/vstsblog/archive/2010/11.aspx
56
http://msdn.microsoft.com/en-us/library/ms242894(v=vs.90).aspx
57
http://msdn.microsoft.com/en-us/library/dd286491.aspx
Page 37 of 109
Planning Guide Defining your Team Strategy
Refer to Hands-on Lab, section Stepping through the planning of a Team Strategy for a walkthrough of this
section
This section is focused on Dave the TFS administrator, Mike the Program Manager, Garry the lead for
development team, Doris the developer, Christine the tester and Paul the database administrator.
Understand Teams 39
Page 38 of 109
Planning Guide Defining your Team Strategy
Understanding Teams
Teams are new to TFS 2012 and they are defined within a Team Project. The Teams concept grew out of the
need of Scrum teams.
Teams allow you to set up a team that has their own Product Backlog to manage from the Team Projects
Product Backlog.
Teams manage their own Area Paths in a Team Project and become a collaboration hub for the team.
The Team ends up being a named group of team members who work on a demarcated area of the Team
Project.
Every Team Project will have a default team created called My Team.
Team Boundaries
The Teams are constrained to their Team Project, with no ability to share data or teams among Team Projects.
The following image illustrates the boundaries of Teams within Team Projects.
Team Constraints
The following table summarizes the constraints associated with Teams:
Feature Constraint
Number of Team There is no hard limit of users within a Team.
Members Large groups are hard to navigate and manage. Try to create smaller sub teams if your
team is large.
Groups larger than 100 will typically not be loaded by Visual Studio Team Explorer.
Team Name A team name must be unique within a Team Project.
Team X can exist in multiple Team Projects, but each will be a different team, sharing
the same name.
Teams within Teams Teams are a flat list of users and TFS Groups.
Table 15 - Team constraints
Page 39 of 109
Planning Guide Defining your Team Strategy
Page 40 of 109
Planning Guide Defining your Team Strategy
Example
Project Background
Using the Trey Research Team Project as a single Team approach, Paul, Doris A, Doris B. and Christine can be
assigned as members on the Trey Research Team. Mike, the Program Manager, will create the tasks in the
Product Backlog and define the sprints. Preliminary assignments will be made by Mike with the Team members
capacity kept in mind. In this scenario, the number of team members is a manageable number and having
multiple teams would add extra complexity that is not needed.
Challenges
Teams allow a team to manage the Product Backlog to a level that is more conducive to a team environment.
The Team functionality takes advantage of the new Product Backlog and Storyboard functionality.
Development Teams that are using Area Path to define a component in their Team Project will not be prevented
from doing so. They will need to understand the planning behind Area Paths starting with TFS 2012.
Team members cannot be assigned multiple roles in a Team within a sprint. For example, Abu who is the build
master cannot be assigned to be a developer or a tester in addition to his build role within a sprint. This is a
limitation of the Team functionality that might be addressed in a future release. A team member can switch roles
between sprints however, not within a sprint.
Recommendations
Keep the teams manageable. Ideally, the team is working on specific functionality or is completely responsible
for an application.
Options
Even Team Projects that dont fit the Scrum or Agile approach can benefit from having a Team defined for it to
provide a quick glimpse into the work to be done, work in progress and work completed. Putting team members
into a Team will get them accustomed to the Team approach and the benefits that will come with it.
Using the Trey Research Team Project as a single Team approach, Paul, Doris A, Doris B. and Christine can be
assigned as members on the Trey Research Team. Mike, the Program Manager, will create the tasks in the
Product Backlog and define the sprints. Preliminary assignments will be made by Mike with the Team members
capacity kept in mind. In this scenario, the number of team members is a manageable number and having
multiple teams would add extra complexity that is not needed.
Page 41 of 109
Planning Guide Defining your Team Strategy
Multiple Team Strategy
You can have a single application in a Team Project. The single application is supported by many developers who
focus on different components of the application or on different release versions. When members of an
application team have a specific focus it is easy to group these people into a team. These team members are not
constrained to a single team. A database administrator is an example of a person who can be on several teams
because they can have different roles on each team.
Example
Project Background
The Consolidated Messenger TFS Project is a web project supported by three development teams (Team A, Team
B and Team C). The web project is a product sold to clients and is being re-designed from classic ASP to MVC. To
keep all three teams involved with the transition to MVC while they continue to support the existing application,
each team spends one month on production support tasks while the other two teams work on the new MVC
functionality. Each month has two sprints for the new development work. The production support month is one
sprint but there might not be a full month of work. If not, the team can work on new functionality for their
upcoming sprints. Each team has a technical lead, a program manager, five developers, a tester, a build master
and a database administrator. The technical lead does development tasks in addition to code reviews and the
code reviews can be done for other teams, not just the team they are assigned to. The program manager tracks
the teams progress, meets with the other project managers and the application owner representative to identify
functionality of the new application as well as help with testing. The developers and database administrator are
dedicated to the tasks that they select during their time in a sprint. However, they can change roles to be a tester
in a different sprint. The build master has a development role during the first sprint of the month and is
dedicated to builds and deployments during the second sprint of the month.
Using the Consolidated Messenger Team Project as a multiple Team approach, each team has a sprint backlog
they create from the product backlog. The team owns their backlog, which is defined by the area/iteration.
During the month that a team is assigned to production support, they can pull items from a future sprint, as long
as those items have been defined for them in that future sprint. For the teams working on new functionality, the
Project Manager takes on testing responsibilities because the first sprint of the month focuses on development
Page 42 of 109
Planning Guide Defining your Team Strategy
and the second sprint of the month focuses on testing. At the start of the sprint, the Project Manager schedules a
review with the team members and then adjusts each team members capacities for that sprint.
Challenges
Deciding to take on work for a future sprint has inherent risk. What happens if you commit to the work and cant
complete it? In the example of the production support team deciding to work on items from a future sprint,
theres the chance that the production code will require changes and that production work will override any
future work they committed to. Now they will have uncompleted future sprint work.
Just like with the single team approach, development teams that are using Area Path to define a component in
their Team Project will not be prevented from doing so. They will need to understand the planning behind Area
Paths, starting with TFS 2012.
Having a person on different teams can lead to capacity issues. The time commitment of one person across
multiple teams can turn into a prioritization issue if it is not properly managed.
Recommendations
Keep the teams manageable. Ideally, the team is working on specific functionality or is completely responsible
for an application.
Options
Even Team Projects that dont fit the Scrum or Agile approach can benefit from the Team approach. By Default
there is always a team created for a Team Project. Using this Team will provide a quick glimpse into the work to
be done, work in progress and work completed
References
Visual Studio ALM Rangers Quick Reference Guidance 58
58
http://vs2010quickref.codeplex.com/
59
http://blogs.msdn.com/b/visualstudioalm/archive/2011/09/20/visual-studio-team-foundation-server-11-developer-preview-what-s-new-for-team-foundation-server.aspx
Page 43 of 109
Planning Guide Defining your Disaster Recovery (DR) Strategy
Understand DR Strategies 44
Understand DR Avoidance 46
Understand DR Recovery 51
Table 17 - Deciding what information is important with disaster recovery planning
Understand DR Strategies
What we understand with Disaster Recovery (DR)
Disaster recovery is the dreaded process of recovering infrastructures and services after disruption by a disaster,
whether man-made60 or natural61. Both are very difficult to predict and prevent, making it important for
organizations to define, implement, and continuously evaluate processes, procedures, and policies to mitigate
the risk to critical infrastructures and services.
Disaster recovery planning typically encompasses preventive, corrective, and detective measures. See:
TFS Installation and Administration Guide 62
Disaster Recovery Planning (Database Engine) 63
Be Prepared: A Guide to SharePoint Disaster Prevention and Recovery 64
60
Man-made disaster examples: war, terrorism, hacking and negligence
61
Natural disaster examples: earthquake, flood and meteor impact
62
http://www.microsoft.com/en-us/download/details.aspx?id=29035
63
http://technet.microsoft.com/en-us/library/ms178128(v=sql.100).aspx
64
http://technet.microsoft.com/en-us/magazine/2005.11.beprepared.aspx
65
http://technet.microsoft.com/en-us/library/ff628971.aspx
Page 44 of 109
Planning Guide Defining your Disaster Recovery (DR) Strategy
As shown in Figure 31, a system can go through four typical states of health. We start with a healthy system
in which all services and hardware function correctly, switch to a compromised system when software or
hardware encounters issues that are not evident to the user, degrade to a faulty system when users notice
errors, and eventually are disrupted by by a disastrous failure.
Its our objective to help you detect the puff of smoke before you are faced with thick plumes of smoke and,
eventually, a raging forest fire that is complex and costly to manage. By pro-actively detecting and managing
issues in your TFS you are minimizing the impact of cost and downtime within your operations, resulting in
productive and happy TFS users.
DR Strategies
The following table summarizes the disaster recovery (DR) strategies that can be considered for TFS and the
recommended66 consideration for use in the Humongous Insurance (HI), Consolidated Messenger (CM) and Trey
Research (TR) customer types.
Strategy Description HI CM TR
Simple In an environment where you are prototyping or are not dependent on TFS
maintaining history, you can adopt a simple DR strategy that is based on re-
installing TFS and checking in the latest known codebase. This codebase is
typically backed up on multiple systems, removable disk drives and/or backups.
Backup and Restore Refer to the TFS Administration Guide 67, section Backing up and Restoring
Your Deployment, to determine the best backup strategy.
TFS 2012: Refer to the Microsoft Visual Studio Team Foundation Server 2012
Power Tools 68 which contain the TFS Backup/Restore tool. This tool can schedule
backups and use the restore wizard to perform a selective or complete restore.
TFS 2012 Update 2: The Backup/Restore PowerTool functionality is now in the
product and referred to as Scheduled Backups. This feature can be accessed
from the TFS Administration Console.
AT Standby Servers To avoid downtime in the event of an Application tier (AT) server failure or AT
server maintenance you can implement a Standby Application tier. Refer to TFS
Installation Guide 69, section Configuring a Standby Application tier for more
information.
66
All of the DR strategies are feasible for all customer types. The recommended consideration is based on practicality, considering complexity and cost of infrastructure and support.
67
http://www.microsoft.com/en-us/download/details.aspx?id=29035
68
http://visualstudiogallery.msdn.microsoft.com/b1ef7eb2-e084-4cb8-9bc7-06c3bad9148f
69
http://aka.ms/tfsinstallguide
Page 45 of 109
Planning Guide Defining your Disaster Recovery (DR) Strategy
Strategy Description HI CM TR
Team Foundation To avoid downtime in an event of a Data tier (DT) server failure or DT server
Cluster maintenance you can implement a Team Foundation Cluster. Refer to TFS
Installation Guide, section Configuring Servers for Team Foundation Cluster
Installation for more information.
Table 18 DR strategies
Backup/Restore PowerTool is not supported on releases where Scheduled Backups is available? Its either you use
NOTE
one or the other. If you upgrade to a build that has Scheduled Backups from one that uses the PowerTool, your
PowerTool settings will be automatically configured into Scheduled Backups and the PowerTool uninstalled as part of
the upgrade.
Refer to Planning the ideal DR environment for TFS, on page 80, for a more detailed planning walk-through.
Understand DR Avoidance
DR Avoidance is about monitoring your system, noticing and dealing with smoke before it turns into a raging
fire, commonly referred to as a disaster.
Microsoft This tool will help you identify missing security updates and How To: Use the Microsoft
Baseline common security misconfigurations in Windows, SQL Server, and Baseline Security Analyzer 70
Security IIS. Used regularly, this tool helps increase system reliability. Microsoft Baseline Security
Analyzer Analyzer 2.2 (for IT
Professionals) 71
70
http://msdn.microsoft.com/en-us/library/ff647642.aspx
71
http://www.microsoft.com/en-us/download/details.aspx?id=7558
Page 46 of 109
Planning Guide Defining your Disaster Recovery (DR) Strategy
Tool Description Reference
Microsoft This tool collects TFS data from your environment and produces a Best Practices Analyzer Tool for
TFS 2010 comprehensive best practice rules report. Use this tool before Team Foundation Server 72
Best installing or upgrading TFS, thereafter on a regular basis, and while Team Foundation Server Power
Practices troubleshooting. Tools December 2011 (TFS BPA
Analyzer download) 73
SCOM TFS Proactive and reactive monitoring of TFS can be done by using the TFS2012: Visual Studio 2012
Monitoring Microsoft System Center Operations Manager (SCOM) together Team Foundation Server
Management with a Management Pack designed for TFS. Monitoring Management Pack
74
Pack Feature Summary (from download link below)
TFS2010: Visual Studio 2010
The monitoring provided by this management pack includes Team Foundation Server
availability and configuration monitoring, performance data Monitoring Management Pack
collection, and default thresholds. You can integrate the monitoring 75
of TFS components into your service-oriented monitoring TFS2008: Visual Studio Team
scenarios. System 2008 Team Foundation
Auto discovery of TFS components. Server Management Pack for
Implements a containment hierarchy, reflecting logical System Center Operations
architecture of the Product. Management 2007 76
Implements a proper health model using Monitors.
Contains tasks, diagnostic and recovery for certain failures.
Provides events that indicate service outages.
Provides alerts that show configuration issues and
connected data source changes.
Verification that all dependent services are running.
Triggers targeted running of BPA against TFS Servers from
Operator Console in TFS 2008 and 2010.
Performance Reads a performance monitor counter log and analyzes it using Performance Analysis of Logs
Analysis of known thresholds. Its a great tool to use when you need to (PAL) Tool 77
Logs (PAL) investigate (potential) performance issues in your environment but
Tool are not familiar with the various performance counters available.
Reports Reports (based on SQL Server Reporting Services) that can be used Report pack: TFS2010:
to evaluate and get a picture of the status of some of the internals Warehouse and Job Service
of your TFS environment. Download the administration reports and Administrator Reports 78
upload them to your TFS Reporting Services environment by Report pack: Administrative
following the instructions found in the links. Report Pack for Team
Foundation Server 2010 79
Blog: Monitoring the TFS Data
Warehouse - FAQ 80
Blog: Data Driven Subscription
Reporting a la Grant 81
72
http://msdn.microsoft.com/en-us/library/ee248630(v=vs.100).aspx
73
http://visualstudiogallery.msdn.microsoft.com/c255a1e4-04ba-4f68-8f4e-cd473d6b971f
74
http://www.microsoft.com/en-us/download/details.aspx?id=35773
75
http://www.microsoft.com/en-us/download/details.aspx?id=6325
76
http://www.microsoft.com/en-us/download/details.aspx?id=14720
77
http://pal.codeplex.com
78
http://blogs.msdn.com/b/granth/archive/2010/02/07/tfs2010-warehouse-and-job-status-reports.aspx
79
http://blogs.msdn.com/b/granth/archive/2010/07/12/administrative-report-pack-for-team-foundation-server-2010.aspx
80
http://blogs.msdn.com/b/granth/archive/2010/07/12/monitoring-the-tfs-data-warehouse-faq.aspx
81
http://blogs.msdn.com/b/willy-peter_schaub/archive/2011/01/31/tfs-integration-platform-data-driven-subscription-reporting-a-la-grant.aspx
Page 47 of 109
Planning Guide Defining your Disaster Recovery (DR) Strategy
Table 20 - Tools that help you troubleshoot and monitor
Processor utilization
Counter Threshold
% Processor Time Should be less than 80% (Minor peaks over 80% are OK)
Memory utilization
Counter Threshold
Disk
Counter Threshold
Network
Counter Threshold
Bytes Total/sec Network Utilization should be less than 40% of the total bandwidth and
anything above 65% is critical.
This is how you calculate % Network Utilization: 83
82
http://technet.microsoft.com/en-us/library/dd262028.aspx
83
http://blogs.technet.com/b/kevinholman/archive/2011/03/02/how-to-collect-performance-data-from-a-script-example-network-adapter-utilization.aspx
Page 48 of 109
Planning Guide Defining your Disaster Recovery (DR) Strategy
Web Performance
Counter Threshold
ASP.NET Applications(*)\Request Execution Time Benchmark your environment when performance is good, to
determine the ideal threshold (as low as possible) for your
TFS Services\Average Response Time
environment.
TFS Version Control\Average Response Time
Table 25 Web performance counters
System
Counter Threshold
Context Switches/sec Should be less than 5000 per processor and more than 10000/processor indicates a constraint
Total Cache Hits Benchmark your environment when performance is good, to determine the ideal threshold (as
high as possible) for your environment.
Table 26 System performance counters
SQL server
Counter Threshold
Average Build This report provides details of the average time taken by successful builds. Average duration for
Duration This will help in identifying and monitoring the build information in tabular successful builds per build
format. definition.
Blocked Field This report shows blocked fields that have conflicts over all Team Project Conflicts across all TPCs
Changes Collections. for fields being blocked.
Administrative Report Pack
for Team Foundation
Server 2010 84.
Build agent This report provides the builds run per hour for a particular TFS Project. Hourly distribution of the
hourly usage of the build agents.
distribution
84
http://blogs.msdn.com/b/granth/archive/2010/07/12/administrative-report-pack-for-team-foundation-server-2010.aspx
Page 49 of 109
Planning Guide Defining your Disaster Recovery (DR) Strategy
Report Description Purpose / Reference Link
Build server This report will provide the build server usage details and details on the Statistics on a build server.
summary number of builds run and their durations.
Cube status This report provides the following information: Used to monitor Analysis
How long is cube processing taking? Cube processing that
How much time elapses between processing jobs? occurs on a regular
How often do the processing jobs run? schedule. Administrative
Do errors occur when the cube is processed? Report Pack for Team
Foundation Server 2010 85.
Execution This report provides a visualization of the load by total execution time on Breakdown of user
time for user the server from and provides details on the users who are putting the execution time.
biggest load on the server.
Execution This report provides a visualization of the load by total execution time on Breakdown of execution
time summary the server from two axes: users and commands. time.
You can use this report when you want to know:
Which commands account for the largest load on the server?
Which tools or users are putting the biggest load on the server?
Job status The Job status report shows the job definitions for the instance and the Job definitions for the
interval theyre set to run on. This is useful for checking to see if a job has instance and the interval
somehow been disabled or changed. The report also shows the Job History. theyre set to run on from
Warehouse and Job
Service Administrator
Reports 86.
Reportable The Reportable Fields report shows all reportable fields in the deployment Administrative Report Pack
fields / of TFS. Administrators of Team Projects can use this report before they add for Team Foundation
Queued fields a reportable field or change the properties of an existing field to prevent Server 2010 87.
changes potential schema-merge conflicts. It lists fields across all collections,
including any fields that are blocked. The Queued Filed Changes report
shows field changes that are queued behind the blocked changes.
Server status - This report serves as a summary of the average response time for two of Performance monitoring.
historical the TFS subsystems: Work Item Tracking and Version Control.
performance You can use this report when you want to know:
trends
How long are users, on average, waiting for a subsystem to
process their request.
Which days of the week are the most critical when it comes to
performance.
Server status - This report provides more granularity about the performance of the server. Performance monitoring.
recent The reports start with a view of the server average response time, looking at
performance the entire picture instead of response time broken down by subsystem. This
trends is followed by charts about version control downloads and average
response time distributions for the same time period.
Use this report when you want to know:
The correlation between degraded server performance and
average response times by subsystem.
85
http://blogs.msdn.com/b/granth/archive/2010/07/12/administrative-report-pack-for-team-foundation-server-2010.aspx
86
http://blogs.msdn.com/b/granth/archive/2010/02/07/tfs2010-warehouse-and-job-status-reports.aspx
87
http://blogs.msdn.com/b/granth/archive/2010/07/12/administrative-report-pack-for-team-foundation-server-2010.aspx
Page 50 of 109
Planning Guide Defining your Disaster Recovery (DR) Strategy
Report Description Purpose / Reference Link
How a large number of downloads affects overall server
performance.
An overall health indicator of the server.
Server Status This report allows administrators a view into which users are not complying Users not using the proxy
- Top users with internal guidelines around proxy usage, which decreases overall server server.
bypassing performance.
proxies
TFS usage This report provides information on the total number of users using the Number of users using a
farm. This will help monitor the load on server. TPC.
Warehouse The warehouse report provides a quick an easy way to find out if an Warehouse and Job
status incremental or full analysis processing is in progress. It also shows any Service Administrator
errors (like warehouse schema conflicts) in the Last Run column. This Reports 88.
report is also useful after an upgrade or when the warehouse needs to be
rebuilt manually. It shows each of the data adapter sync jobs for each
collection and their current status. During normal operation, these will run
very quickly as data changes in the operational stores, and probably always
appear Idle. It will also show any errors from previous job executions in
the Last Run column.
Average This report provides the average response for requests made by users. This Performance monitoring.
response time report will help in understanding the performance of the farm. As a
standard, a lower average response time signifies a good health of the
farm.
SQL This report provides the details of the connectivity issues between the Report to Check the SQL
connection Application and Data tiers. As a standard, a lower number of failures issues 89
failures/sec signifies good farm health.
CPU This report provides the details of the CPU utilization of the Application tier. Monitor CPU Utilization on
utilization As a standard, a lower number of CPU utilization signifies that the system is App and Data tier
not under load and can support additional users.
Available This report provides details of the RAM utilization of the application and SQL and TFS App tier
memory Data tier. This report can be used to monitor the free RAM in the servers, memory usage
which is important for smooth operation of the farm.
Requests/sec This report provides details on the user load on the system. A higher RPS Performance monitoring.
ond signifies a higher load on the system.
Table 1 - Reports for Monitoring TFS health
DR Recovery Walkthroughs
If you are in the unfortunate position of being confronted with a fire (disaster), we have created walk-throughs
for you to step through the recovery process.
88
http://blogs.msdn.com/b/granth/archive/2010/02/07/tfs2010-warehouse-and-job-status-reports.aspx
89
http://blogs.msdn.com/b/granth/archive/2008/11/07/querying-perfmon-data-from-sql.aspx
Page 51 of 109
Planning Guide Defining your Disaster Recovery (DR) Strategy
Disaster Recovery Scenario See page
AT Failure 87
Build Failure 92
Proxy Failure 91
Table 28 - Disaster recovery walkthroughs
Page 52 of 109
Planning Guide Large and Complex Project considerations
This section focuses on Dave the TFS administrator, Jane the infrastructure specialist, and Garry the lead for
development teams.
Page 53 of 109
Planning Guide Large and Complex Project considerations
The definition of a large and potentially complex project varies from organization to organization. Figure 33
shows four typical large project definitions we have encountered:
1. Projects with a large number of concurrent users (100+), which result in spikes of builds/day and place
pressure on services, such as Reporting Services.
2. Projects that involve a number of business units with separate responsibility boundaries and cross-
business unit development within a single organization.
Page 54 of 109
Planning Guide Large and Complex Project considerations
What are the challenges of large projects?
As projects grow in size and complexity, they introduce additional challenges such as:
Builds become more numerous, complex and costly to administrate. As a project gets larger and especially
as cross-team development scenarios emerge, administrators have to deal with:
o Dependencies between Team Project Collections, Branches and Build servers
o Increased builds and long-running builds, which can introduce unproductive queuing and loading of the
build servers.
See Visual Studio ALM Rangers Build Customization Guidance 90 for more information.
Branching and Merging becomes more complex, introducing time and resource consuming forward
(maindev) and reverse (devmain) integration, which usually emerges as build failures, missed milestones
and features that fail to propagate to the Main branch.
See Visual Studio ALM Rangers Branching and Merging Guidance 91 for more information.
Teams and dependencies between boundaries of responsibility become more difficult to manage.
o The size and number of teams typically start to grow with larger projects and introduce collaboration
and dependencies, which must be carefully considered to avoid complexity and competition for
resources.
o Distributed and virtual team ecosystems emerge as team boundaries extend beyond on-premises
environments, which introduce unique infrastructure, management and collaboration challenges.
o The choice of Infrastructure, Team Project Collection, Team Project and Team strategies start to play a
pivotal role. Ensure that you plan early and observe and evolve continuously.
See Visual Studio ALM Rangers - Reflections on Virtual Teams 92 to get a view of managing virtual teams.
Isolation is total between Collections. Process Isolation is configurable at Team Project level. Almost
Templates, Reports, Work Items, Queries, Version everything is possible to share, including Process Templates,
Control, Build Services are inaccessible over the Reports, Work Items, Queries, Version Control and Build
collection boundary. Services.
Separate security setting for each Collection that Top level security is set on Collection and propagated to the
defines the top level security for each contained Team Team Projects. Each Team Project could define modifications of
Project. / additions to the top level security.
Table 30 Clear isolation level difference between multi and single collection strategy
90
http://go.microsoft.com/fwlink/?LinkID=230938
91
http://go.microsoft.com/fwlink/?LinkID=230936
92
http://blogs.msdn.com/b/willy-peter_schaub/archive/2011/09/07/visual-studio-alm-rangers-reflections-on-virtual-teams.aspx
Page 55 of 109
Planning Guide Large and Complex Project considerations
Single versus Multiple Team Project Collection Strategies
The advantages of using Multiple or Single Collections
Possible to implement a more complex isolation strategy. Cross Team Project sharing or linking of Work Items,
Queries and Reports are possible.
Starting, stopping, backups are possible on individual Cross Team Project sharing of content in Version Control
Collections. It affects only Team Projects in that Collection. is possible (Move/Branch/Merge).
Possible to distribute Collection databases across SQL Server Cross Team Project working is possible.
instances.
Looking from the opposite view, we can of course also see some disadvantages:
The disadvantages of using Multiple or Single Collections
No cross Collection sharing of Work Items, Queries and Reports. Single database for all Team Projects data.
No cross Collection Version Control (Move/Branch/Merge). Restore operations of database affects all Team
Projects.
No cross Team Project work is possible if Team Projects are Each Collection has its limited capacity of Team
located in different Collections. Projects.
The need for isolation is preferred The need for collaboration is preferred
There are many Team Projects There are few Team Project
There are many users or large teams There are few users or small teams
You have a complex organization of projects in projects etc. You have a simple organization
Table 33 When to use multiple or single collections
As we have already seen there is no clear definition of when to use a specific approach.
Page 56 of 109
Planning Guide Large and Complex Project considerations
What we can say with certainty is that we typically have some high level grouping where, in most cases, some
isolation is required. For these groups we recommend using separate Collections.
In the mid-level, we typically find items that in some way are related to each other. This could be Clients,
Releases, Products or Business Units. At this level we recommend using separate Team Projects.
At the lowest level, we typically find some logical units that, combined, add up into one of the mid-level items.
This is the level where we recommend using Components (or Area if backward compatibility is used).
Page 57 of 109
Planning Guide Large and Complex Project considerations
Separate Team Projects are also used for each project. Team Projects contain branches from a collection of
different products master Team Projects that are included in that specific project. Each Team Project also keeps
its own backlogs and reports grouping the project work together.
Each master Team Project is connected to the different projects and maintenance queries and reports. Having a
master Team Project enables follow up on the lifecycle of the product.
The administration of members and their team assignments is easily maintained on collection level and / or
Team Project level. The members could be allowed read access to the whole collection and at the same time
have extended rights to each specific Team Project that they are specifically involved in.
To increase availability, a centralized instance of TFS is used in combination with TFS Proxies on locations where
bandwidth problems exist.
The build environment consists of centralized Build Controller / Agents with the drop shares placed at local
locations. This ensures a standardized way of building as fast as possible at the same time as the build output is
delivered to local locations for testing and deploying.
To keep a high level of isolation, each city has its own collection. By using Multiple Team Project Collections, it is
possible to back up and restore each citys database without affecting the other cities.
Its also possible to ensure that only consultants in a specific city are given access to that citys different clients.
The other cities and clients are totally invisible from the outside.
Each client is kept in a separate Team Project with Areas defined for the different projects and maintenance.
Its easy for a single consultant to participate in several teams, working for several clients concurrently.
Its easy to share different artifacts between the clients that the consultants are working with.
Each city is responsible for its own build environment, making it possible to have a custom made build process
that covers their own clients specific needs.
Page 58 of 109
Planning Guide Large and Complex Project considerations
Migrating and Coexisting with Legacy Systems
Basic Decisions
We recommend dealing with the legacy scenario that is based on the Team Project guidance; see page 26 for
details. In essence:
If the legacy code requires different security models, different process models or different ownership, we
consider a separate Team project.
If we split based on ownership:
o If the same team is working the legacy code, consider one Team Project for all legacy code.
o If different teams are working on different parts of the legacy code, for example different legacy
solutions, consider one Team Project for all legacy code and using Teams to segregate teams.
o If different teams from different organizations are working on different parts of the legacy code,
for example different legacy solutions, consider a Team Project per organizational business unit.
Examples
When legacy code is included in the solution, we recommend that it is maintained by a separate team in its own
Team Project. This ensures that the legacy code will not remain entwined in the solution code base once it is
retired, because it is not a trivial task to split a Team Project and its artifacts into multiple Team Projects.
Figure 37 - Single TPC & TP, with legacy code - Trey Research
Page 59 of 109
Planning Guide Large and Complex Project considerations
Multiple Team Project strategy Humongous Insurance
As outlined on page 27 and in particular page 34, Humongous Insurance might decide for the Multiple Team
Project strategy, if they are developing several independent solutions.
As with the previous example, we recommend that legacy code is maintained by a separate team in its own Team
Project. This ensures that the legacy code will not remain entwined in the solution code base once it is retired,
because it is not a trivial task to split a Team Project and its artifacts into multiple Team Projects. In the case of
Humungous Insurance we have, as shown in Figure 38, two legacy and two new solution code bases, which
results in the following possible Team Project design:
Figure 39 - Single TPC & Multi-TP, with legacy code Humungous Insurance: example 1
If, for example, we had one new solution, based on three separate legacy code bases, the Team Project design
would change as follows:
Figure 40 - Single TPC & Multi-TP, with legacy code Humungous Insurance: example 2
Page 60 of 109
Planning Guide Large and Complex Project considerations
Multiple Team Project Collection strategy and Legacy Code
Figure 41 - Multiple TPC & TP, with legacy code Consolidated Messenger
93
http://vs2010quickref.codeplex.com/
94
http://go.microsoft.com/fwlink/?LinkID=230936
95
http://go.microsoft.com/fwlink/?LinkID=230938
96
http://blogs.msdn.com/b/willy-peter_schaub/archive/2011/09/07/visual-studio-alm-rangers-reflections-on-virtual-teams.aspx
Page 61 of 109
Planning Guide Questions & Answers
Security Isolation
What are the possible options to restrict access to the SQL Analysis Services Cube that TFS utilizes in a
multi Team Project Collection scenario?
It depends on what is meant by restricting access. If the users are not members of the TFSWarehouseDataReader
role in the Tfs_Analysis database they will not be able to query the cube directly (well, unless they are admins).
For Reporting Services, you would need to remove or not grant them permissions in the project collection
folders on the reporting website that you dont want them to see. 97
Grant Team Members Access to the Analysis Services Cube 98
If the question is How do I prevent users from collection A from seeing warehouse data from collection B? we
do not believe there is a way to apply permissions at the collection level in the cube. Permissions are all or
nothing.
When renaming a folder which is included in a Continuous Integration build definition, builds may fail
Context: This question is not really a question but more of informational type. We had a Team Project where
they renamed a folder included in the workspace for a Continuous Integration Build Definition. A funny thing
97
ALMTalk, George Archer, Senior Support Escalation Engineer, Microsoft
98
http://msdn.microsoft.com/en-us/library/ms244642.aspx
Page 62 of 109
Planning Guide Questions & Answers
happened because all of a sudden the TFS lost control of that build's workspace and reverted back to the version
control root so every time someone checked in something in another Team Project; it triggered the erroneous
build which of course failed because the folder defined in the workspace didn't exist anymore. As a result, we
ended up with approximately 6000 failing builds triggered during a 24 hour period.
Answer: The solution we decided to try was to create a PowerShell-script that runs as a task to make a sanity
check of all the build definitions. We wanted to make sure that the workspaces that wed defined really exist.
Otherwise, the PowerShell-script will make sure that erroneous build definition isnt triggered by check-ins. A
future update of this script might also include emailing the project admins of that particular Team Project about
the change of the Build Definition.
Doing a solution like this is pretty simple because the Object Model of TFS is so script friendly.
Page 63 of 109
Planning Guide Real World Reference Stories
Page 64 of 109
Planning Guide Real World Reference Stories
Challenges
The technical challenge in this scenario is the migration from the old server to the new server without loss of
data. The best way depends a lot on the specific scenario. For migration scenarios we recommend that you read
the Upgrade Guidance 100 and Installation Guidance 101. However, for this Team Project Guidance, the challenge
would have been to find out what best suits infrastructure for this company in the first place. A reinstallation of
TFS after a short period should never have been necessary. Good planning should be part of every TFS
installation regardless of whether the installation is big or small.
Lessons learned
Plan your installation no matter if you are planning for a small company or an enterprise.
Think about possible side effects when you share your TFS servers for multiple applications. This includes, for
example, version updates and service packs.
If you are not sure what to do, try to find answers using official installation instructions or guidance
documents provided by the ALM Rangers, such as:
o TFS Installation Guidance
o Team Project Guidance written by the ALM Rangers (this document)
Ask TFS consulting experts if you are not sure what to do. There are plenty of companies out there who are
willing to help you get your system going. If youre unsure about who to ask, contact Microsoft and ask for a
recommendation.
Keep in mind that money spent for planning will pay out twice later.
99
http://tfsintegration.codeplex.com/
100
http://go.microsoft.com/fwlink/?LinkID=230948
101
http://go.microsoft.com/fwlink/?LinkID=231279
Page 65 of 109
Planning Guide Real World Reference Stories
Challenges
In this scenario, there are several points to be addressednot all of them directly connected, but most of them
somehow related to the structure of TFS. Its never easy to point out something as wrong from a distance. Still,
here are some things to consider:
Lessons Learned
Is the mapping of Team Projects to Teams optimal or could you decrease complexity using less Team
Projects?
Do you have a branching strategy that you can rely on? If you are not sure or feel its too complex, try to
follow the ALM Rangers Branching Guide 103.
Do you need more build servers? Could you improve feedback time and reduce complexity by using
additional build agents or by a redesign of your build infrastructure? You might find an answer in the ALM
Rangers Build Customization Guidance 104.
102
Note that the use of tags isnt going to magically make queue times shorter.
103
http://go.microsoft.com/fwlink/?LinkID=230936
104
http://go.microsoft.com/fwlink/?LinkID=230938
Page 66 of 109
Planning Guide Real World Reference Stories
Challenges
So whats wrong here? Nothing!
As long as it works and you can handle the complexity. But thinking about what might come up in the future,
theres always room for improvements:
Lessons Learned
Think about dispensing the agents on several servers, if you are running into troubles regarding shared OS
resources.
Think about the source structure, following some recommendations in the ALM Rangers Branching and
Merging Guidance 105.
Think about if you really need to drop every buildfor example, it might be a good idea to throw away build
results from a successful gated check-in build to save the time for the file copy and the disk space. It might
be a better idea to just keep results of one daily build (for example, Nightbuild) and manually triggered
builds.
105
http://go.microsoft.com/fwlink/?LinkID=230936
Page 67 of 109
Planning Guide Real World Reference Stories
Challenges
Running an infrastructure all across the globe isnt easy. TFS brings the option to install a proxy that will be a very
good solution to a lot of issues regarding performance in slower networks. But theres more to consider.
Lessons Learned
Think about which data is typically being transferred when a developer does his or her daily business. Typically this isnt
necessarily a full Get but might be only an Update Get.
Teach your developers to use the Cloaking feature Visual Studio / TFS offers. This will reduce execution time of Get
for the developer by explicit exclusion of files to be transferred.
Consider using Build Servers that query a TFS proxy instead of the TFS app tier for data. The TFS proxy only caches
source code filesall a build server needs is source code files. Seems like a perfect match, doesnt it?
Think about physical locations of your infrastructure. Its always a good idea to stay as close to your team as possible.
Running multiple teams across the globe might result in several build servers spread across the globe.
Think about which data transfer happens most often and where the amount of data being transferred is biggest. Start
looking for improvements there. In the scenario described, the builds triggered from Asia happened often and
transferred all source code, which makes it a good area for optimization.
Page 68 of 109
Planning Guide Real World Reference Stories
Challenges
The scenario described here shows how a company gets rid of an unpleasant infrastructure by refactoring
concurrently with already required installation efforts. In scenarios like these, where you could continue to work
the way you are used to, but feel that something is wrong, theres always the question of when you will find the
time to refactor. Using the upgrade process can be a very effective decision.
Lessons Learned
The best way to handle situations like these is to avoid them in the first place.
If you are in such a situation, try to find a good point in time. The upgrade from TFS 2010 to TFS 2012 might
suit perfectly. You can run a full test upgrade if you have the chance to work on separate hardware. If you
dont have additional hardware you could try running on virtual machines. On your test installation, you can
easily test your refactoring.
Try to address all pain points that you are experiencing. Some of them like the usage of AD groups might
seem very small but have a very high impact on administration.
Keep an eye on the number of Team Projects.
Keep an eye on the permissions. Its not a good idea to give administrative permissions in TFS to everyone in
your company, even though you might know and like all of the people.
Create a separate instance of TFS on separate hardware or at least a separate Team Project Collection where
people are allowed to try something out. You can be much less restrictive with permissions on that system.
If you are refactoring and restructuring why not drop data you wont need any longer? Try to be sure that
old data wont be needed any longer. Even though there are easy ways to upgrade an existing system to the
next version without the loss of data, in situations where the old system is messed up, it might be better to
start over on a clean system and to import only relevant data, such as the latest version of source code
without the history. If you feel that this will probably work for you, its a very easy mechanism to clean up
your system completely and to get the chance to start over.
Page 69 of 109
Planning Guide Real World Reference Stories
Walkthrough
1. Hotfix for time zone issue related to localization was received from Microsoft.
2. Verified that full backup of AT and DT from previous night were available.
3. Patch applied to production environment.
4. AT partially functioning. Build subsystem stopped working.
5. Issue became a fire disaster!
Reaction
1. Discussions between support and Vladimir recommended an uninstall and re-install of the hotfix.
Performed however; could not bring TFS back online; AT down.
2. Vladimir decided to re-install TFS ATs OS from scratch.
3. Discussions between support and Vladimir recommended a restore of the configuration database.
4. Application tier was back, but only partially functional.
5. Day 1 passed and operational environment was still not functional.
6. Decided to start using server without build system and fix it later.
7. Check-ins were uncontrolled; recommendation to manage user base.
8. Identified that stored procedure wasnt updated as desired during initial hotfix installation. That change was
stored in project collection DB and it wasnt rolled back. It is too late to roll it back because users already did
check-ins.
9. Received corrected stored procedure from support and finally fixed the issue.
Outcomes
1. Changed backup plan to be more granular (from twice daily incremental to hourly).
2. Perform periodic testing in staging environment; duplicate the environment frequently.
Page 70 of 109
Planning Guide Real World Reference Stories
Detailed Message: TF221122: An error occurred running job Build Warehouse Sync for team project
collection or Team Foundation server DefaultCollection.
Exception Message: A severe error occurred on the current command. The results, if any, should be
discarded. (type SqlException) SQL Exception Class: 11 SQL Exception Number: 0 SQL Exception Procedure:
SQL Exception Line Number: 0
SQL Exception Server: DT
SQL Exception State: 0
SQL Error(s):
Exception Data Dictionary:
HelpLink.ProdName = Microsoft SQL Server HelpLink.ProdVer = 10.50.1600 HelpLink.EvtSrc = MSSQLServer
HelpLink.EvtID = 0 HelpLink.BaseHelpUrl = http://go.microsoft.com/fwlink HelpLink.LinkId = 20476
Exception Stack Trace: at System.Data.SqlClient.SqlConnection.OnError(SqlException exception,
Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader
dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlDataReader.HasMoreRows()
at System.Data.SqlClient.SqlDataReader.ReadInternal(Boolean setTimeout)
at Microsoft.TeamFoundation.Framework.Server.ObjectBinder`1.TryMoveNext()
at Microsoft.TeamFoundation.Build.Server.BuildInformationMerger.TryMergeNext()
at Microsoft.TeamFoundation.Build.Server.CommandQueryChangedBuilds.ContinueExecution()
at Microsoft.TeamFoundation.Build.Server.CommandQueryChangedBuilds.Execute(IList`1 teamProjects,
DateTime minChangedTime, BuildStatus statusFilter, IList`1 informationTypes)
at
Microsoft.TeamFoundation.Build.Server.TeamFoundationBuildService.QueryChangedBuilds(TeamFoundationReque
stContext requestContext, IList`1 teamProjects, DateTime minChangedTime, BuildStatus statusFilter,
IList`1 informationTypes)
at Microsoft.TeamFoundation.Build.Adapter.TeamBuildWarehouseAdapter.ProcessAndUploadBuildData()
at Microsoft.TeamFoundation.Build.Adapter.TeamBuildWarehouseAdapter.MakeDataChanges()
at
Microsoft.TeamFoundation.Warehouse.WarehouseSyncJobExtension`1.MakeDataChanges(TeamFoundationRequestCon
text requestContext, TeamFoundationJobDefinition jobDefinition, String& resultMessage)
at
Microsoft.TeamFoundation.Warehouse.WarehouseSyncJobExtension`1.RunInternal(TeamFoundationRequestContext
requestContext, TeamFoundationJobDefinition jobDefinition, DateTime queueTime, String& resultMessage)
at Microsoft.TeamFoundation.Warehouse.WarehouseJobExtension.Run(TeamFoundationRequestContext
requestContext, TeamFoundationJobDefinition jobDefinition, DateTime queueTime, String& resultMessage)
Page 71 of 109
Planning Guide Real World Reference Stories
processing aborted, stating that there is a database inconsistency in the collection database. Now that we had
the real error message, we were able to address the original issue.
To make things even worse, the log also showed the following message:
repair_allow_data_loss is the minimum repair level for the errors found by DBCC CHECKDB
(Tfs_DefaultCollection, repair_allow_data_loss).
Therefore, we were not able to recover the consistency errors introduced into the collection database. Because
we cannot guarantee that a repaired collection database is consistent from a logical point of view, we strongly
recommended to roll back the whole collection database in order to get a consistent state again. The client also
got in touch with Microsoft support to find out whether a repair is a supported scenario. Microsoft support also
disagreed and recommended that we recover the last valid database backup.
Recovery
Next, we recovered the old backup from the tape and restored that last good state of the TFS farm. We also tried
to recover at least the last information:
Head Revision of the source control
Recent Work Item changes
The client performed backups of the TFS system using a standard SQL backup plan. The really bad part is that the
client also configured a regular job to check the consistency of the database as part of the backup maintenance
plan. However, nobody monitored the result of the step because all the backups were written properly.
Otherwise, they would have realized the error much sooner.
Learning
This case shows once again how critical and essential monitoring server systems really is. This not only includes
operational parameters like are the TFS web services available or is the SharePoint site online. it also has to
cover the database and the integrity of it.
Page 72 of 109
Planning Guide Appendix
Appendix
Enabling and using TFS Logging Examples
When you plan for an upgrade, a migration, or even a pristine new environment, it is often invaluable to gather
and analyze current and planned traffic with TFS and associated systems. This appendix introduces two of the
options for logging clients that are available to you: Visual Studio and Debug View.
<system.diagnostics>
<switches>
<add name="TeamFoundationSoapProxy" value="4" />
<add name="VersionControl" value="4" />
</switches>
<trace autoflush="true" indentsize="3">
<listeners>
<add name="perfListener"
type="Microsoft.TeamFoundation.Client.PerfTraceListener,
Microsoft.TeamFoundation.Client, Version=11.0.0.0,
Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"/>
<add name="myListener"
type="Microsoft.TeamFoundation.TeamFoundationTextWriterTraceListener,
Microsoft.TeamFoundation.Common, Version=11.0.0.0,
Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"
initializeData="c:\Temp\VS2011Log.log" />
</listeners>
</trace>
</system.diagnostics>
Page 73 of 109
Planning Guide Appendix
Visual Studio 2010 Users
Run notepad as an administrator to open the file C:\Program Files (x86)\Microsoft Visual Studio
10.0\Common7\IDE\devenv.exe.config to enable tracing for Visual Studio 2010.
Configure the following just before the </configuration>:
<system.diagnostics>
<switches>
<add name="TeamFoundationSoapProxy" value="4" />
<add name="VersionControl" value="4" />
</switches>
<trace autoflush="true" indentsize="3">
<listeners>
<add name="perfListener"
type="Microsoft.TeamFoundation.Client.PerfTraceListener,
Microsoft.TeamFoundation.Client, Version=10.0.0.0, Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a"/>
<add name="myListener"
type="Microsoft.TeamFoundation.TeamFoundationTextWriterTraceListener,
Microsoft.TeamFoundation.Common, Version=10.0.0.0, Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a"
initializeData="c:\Temp\VS2010Log.log" />
</listeners>
</trace>
</system.diagnostics>
<system.diagnostics>
<switches>
<add name="TeamFoundationSoapProxy" value="4" />
<add name="VersionControl" value="4" />
</switches>
<trace autoflush="true" indentsize="3">
<listeners>
<add name="perfListener"
type="Microsoft.TeamFoundation.Client.PerfTraceListener,
Microsoft.TeamFoundation.Client, Version=9.0.0.0, Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a"/>
<add name="myListener"
type="Microsoft.TeamFoundation.TeamFoundationTextWriterTraceListener,
Microsoft.TeamFoundation.Common, Version=9.0.0.0, Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a"
initializeData="c:\Temp\VS2008Log.log" />
</listeners>
</trace>
</system.diagnostics>
Page 74 of 109
Planning Guide Appendix
5. Take note of the DialogPerfListener Window, shown in Figure 42, which launches in the background,
together with Visual Studio.
If you want to see how performance matches up to what others are seeing, you can highlight the data lines and use
NOTE
Control + C to copy them to the clipboard. Then you can compare them to the run times for the same project.
6. (Optional) Click Save All to log the data into a text file, as shown in Figure 43.
Page 75 of 109
Planning Guide Appendix
Using Debug View as Logging Client
Another option is to monitor TFS connection and processing traffic and use the DebugView application as the
logging client. The application can be downloaded from http://technet.microsoft.com/en-
us/sysinternals/bb896647.aspx or run from http://live.sysinternals.com/Dbgview.exe.
The commands are logged as shown in Figure 44 and optionally logged to a text logfile as shown in Figure 45.
References
Microsoft TechNet Article on DebugView: Debug View 106
Bart Wullems article on Tracing TFS Data: Tracing TFS Data 107
Buck Hodges article on capturing performance information in Visual Studio: How to Measure
Performance Using the Web Service Performance Dialog 108
Ed Hintzs article on TFS Client Tracking: TFS Client Tracing 109
106
http://technet.microsoft.com/en-us/sysinternals/bb896647.aspx
107
http://bartwullems.blogspot.com/2010/10/tracing-tfs-data.html
108
http://blogs.msdn.com/b/buckh/archive/2006/09/25/performance-dialog.aspx?wa=wsignin1.0
109
http://blogs.msdn.com/b/edhintz/archive/2007/03/30/tfs-client-tracing.aspx
Page 76 of 109
Planning Guide Appendix
TFS Services
TFS Services Counters for Services in TFS
Total Number of Failed retry sequences Number of failed retry sequences
Total Number of Throttling Events Number of Throttling events with SQL
Total Number of SQL Batches Number of interactions with SQL (number of batches)
Current SQL Executions/Sec Current SQL Executions is the rate at which SQL Queries are being
performed
Current SQL Notification Queries/Sec Current SQL Notification Queries is the rate at which Queries for SQL
notifications are being performed
Current Task Executed/Sec Current Task Executed is the rate at which tasks are being executed
Active Team Project Collection Service Hosts Number of active Team Project Collection service hosts
Active Application Service Hosts Number of active application service hosts
Active Deployment Service Hosts Number of active deployment service hosts
Current SQL Execution Retries/Sec Current SQL execution retries is the rate at which SQL command
execution is being retried
Current SQL Connection Retries/Sec Current SQL connection retries is the rate at which SQL connection
retries are being attempted
Current SQL Connection Failures/Sec Current SQL connection failures is the rate at which SQL connection
attempts are failing
Average SQL Connect Time Average time needed to open a new SQL connection
Active SQL Connections Number of SQL connections in any state used by TFS services
Average Response Time Average Response Time is the time, on average, that the server took to
process a single request
Page 77 of 109
Planning Guide Appendix
TFS Services Counters for Services in TFS
Current Requests/Sec Current Requests/Sec is the rate at which requests are being processed
by the server
Current Server Requests Current Server Requests indicates the number of currently active
requests being processed by the server
Table 36 - Perf Counters: TFS services
Current Server Requests Current Server Requests indicates the number of currently active requests being processed
by the Version Control service
Current File Downloads/Sec Current File Downloads/Sec is the rate that files are being downloaded from the Version
Control service
Current File Downloads Current File Downloads indicates the number of files currently being downloaded from the
Version Control service
Current File Uploads/Sec Current File Uploads/Sec" description="Current File Uploads/Sec is the rate that files are
being uploaded to the Version Control service
Current File Uploads Current File Uploads indicates the number of files currently being uploaded to the Version
Control service
Write Access Elevations/Sec Write Access Elevations/Sec is the rate at which requests are elevated to
Write access
ReadLatest Access Elevations/Sec ReadLatest Access Elevations/Sec is the rate at which requests are
elevated to ReadLatest access
Active GetQueryAccessControlList Requests The number of access control list query requests currently executing
Active GetStoredQueries Requests The number of stored queries requests currently executing
Active GetStoredQuery Requests The number of stored query requests currently executing
Active GetMetadata Requests The number of cache updates currently executing
Active Update Requests The number of updates currently executing
Active GetWorkitem Requests The number of work Item requests currently executing
Active Paging Requests The number of paging requests currently executing
Active Query Requests The number of queries currently executing
Table 38 - Perf Counters: TFS work Item tracking
Page 78 of 109
Planning Guide Appendix
TFS Lab Management
TFS Lab Management TFS Lab Management Performance Counters
Current Requests Number of Lab Management requests currently active inside server
Requests/Sec Number of Lab Management requests processed per second
Current Lab Environment Creations Number of lab environments TFS is creating on this server
Current Operations Number of lab environment or template operations (create, start,
stop, snapshot, delete, and so forth) in progress on this server
Total Operations Number of lab environment or template operations (create, start,
stop, checkpoint, delete, and so forth) completed since the last
reboot. This number also includes unsuccessful operations
Total Lab Environment Creations Number of lab environments created since the last reboot
Total Lab Environment Creation Failures Due To Lack Number of unsuccessful lab environment creations because of a
Of Resources lack of resources since the last reboot
Powershell Cmdlets/Sec Number of PowerShell cmdlets executed per second
Runspaces Created Number of runspaces created
Kvp Data Cache : # entries Number of entries in Kvp data cache
Kvp Data Cache : hit ratio Hit Ratio of Kvp cache
Kvp Data Cache : hit ratio base counter Hit Ratio Base counter of Kvp cache
Kvp Data Cache : # trims Number of entries removed either invalid or to make space for
others in Kvp cache
VMMS Cache : # entries Number of entries in the SCVMM cache
VMMS Cache : hit ratio Hit ratio of the SCVMM cache
VMMS Cache : hit ratio base counter Hit ratio base counter of the SCVMM cache
VMMS Cache : # trims Number of entries removed either they were not valid or to
allocate space for other entries in the SCVMM cache
CS Cache : # entries Number of entries in 'computer system objects' cache
CS Cache : hit ratio Hit Ratio of 'computer system objects' cache
CS Cache : hit ratio base counter Hit Ratio Base counter of 'computer system objects' cache
CS Cache : # trims Number of entries removed either invalid or to make space for
others in 'computer system objects' cache
VMM Object Cache : VM & Template : # entries Number of entries in the VM and Template cache
VMM Object Cache : Location : # entries Number of entries in the Host, Host Group, and Library Share
caches
VMM Object Cache : Snapshot : # entries Number of entries in Snapshot cache
VMM Object Cache : Task : # entries Number of entries in Task cache
VMM Object Cache : Profile : # entries Number of entries in the Hardware profile, OS profile, and User
role profile caches
Table 39 - Perf Counters: TFS Lab Management
Page 79 of 109
Planning Guide Appendix
DR Recovery Walkthroughs
Walkthrough: Planning the ideal DR environment for TFS
Planning
Step Description Details Done
1 Analyze Server Strategies Refer to page 9 to decide which server strategy, for example on-premises,
is most ideal for the requirements and IT environment.
2 Analyze DR Strategies Using Table 19 on page 46, determine the best DR strategies for the TFS
Server strategy.
3 Select a DR Strategy See the following tables, based on the selected DR Strategy:
Table 41 Walkthrough: Planning the ideal DR environment for
TFS Simple Strategy, page 80.
Table 42 Walkthrough: Planning the ideal DR environment for
TFS Backup Strategy, page 81.
Table 43 Walkthrough: Planning the ideal DR environment for
TFS Standby Strategy, page 81.
Table 44 Walkthrough: Planning the ideal DR environment for
TFS - Cluster Strategy, page 82.
5 Infrastructure Diagram Create an infrastructure / topology diagram, including machine names and
roles, that can be used as an invaluable reference during disaster recovery.
6 Evaluate and refine the DR Evaluate the planned DR environment, refine the recovery process, and
Strategy train the support staff.
7 Evangelize your DR Raise the awareness of the selected DR Strategy and ensure that the
Strategy escalation process and the contact information are accessible to all
stakeholders.
Table 40 Walkthrough: Planning the ideal DR environment for TFS
Simple Strategy
When you opt for the simple strategy, historical data such as version control or work item history, is not
important. This strategy is typically used in evaluation, prototyping, or training environments, with limited, if any,
service level agreements and disaster recovery requirements.
Option Description Details Done
1 Physical environment Consider a single server topology to keep installation, configuration, and
administration as simple as possible.
2 Virtualized environment The use of a virtual machine allows recovery from snapshots or backups
and offers straightforward disaster recovery.
Table 41 Walkthrough: Planning the ideal DR environment for TFS Simple Strategy
Page 80 of 109
Planning Guide Appendix
Backup Strategy
The objective of the backup strategy is to prevent data loss due to hardware and/or software failure.
Step Description Details Done
1 Understand TFS Server Read Backing up and Restoring Your Deployment 110.
Backup and Restore
2 Define a backup and Define a backup, restore, and validation plan for your deployment topology for
restore plan TFS. This should be applicable for SharePoint Products, SQL Server Reporting,
SQL Server Analysis Services, TFS Build, and VS Test Controllers.
3 Do you need to restore To be able to restore to a different TFS instance, you need to back up and
to a different TFS restore the Tfs_Configuration database as well.
instance? If you need to move a collection to another server, the recommended way is to:
(a) Stop and detach your Team Project Collection, which copies the shared
data from Tfs_Configuration database into the collection database and
produces a portable collection.
(b) Back up the collection.
(c) Re-attach Team Project Collection.
See https://gist.github.com/2865415 for a handy script that automates detach,
back up, and re-attach. This process takes recovery time, which results in
outages for users of the affected collection. Account for this recovery time in
your DR planning and service level agreements.
4 Do you need a latest Often software development teams will ask for a shadow of the latest version
shadow version control data on a file server to use for quick reference or backup during backup
control copy? outage times. You can automate the get latest using PowerShell and create a
shadow update process, which can optionally be run before scheduled outages.
See Get latest files from TFS using PowerShell 111 for an example.
5 Validate^2 Validate the backup, restore, and recovery processes before switching on
production and whenever an infrastructure change is planned.
Table 42 Walkthrough: Planning the ideal DR environment for TFS Backup Strategy
Standby Strategy
The objective of the standby strategy is to avoid outages due to the failure of a TFS Application-tier server.
Option Description Details Done
1 Warm Review Activating a Fail-Over Application-Tier Server 112, which covers the how to
standby AT set up and activate a warm standby AT server.
Server
When the primary server fails, you have to complete manual activation steps. This
implies that there will be an activation outage that ranges from brief to substantial.
2 Load Review How to: Create a Team Foundation Server Farm (High Availability) 113 , which
balanced AT covers the use of Network Load Balancing 114. Because there is no need for manual
Server intervention, there will be no outage when an AT server fails.
Table 43 Walkthrough: Planning the ideal DR environment for TFS Standby Strategy
110
http://msdn.microsoft.com/en-us/library/bb552295(v=vs.100).aspx
111
http://blogs.infosupport.com/getting-latest-files-from-tfs-using-powershell/
112
http://msdn.microsoft.com/en-us/library/ms252486(v=VS.90).aspx
113
http://msdn.microsoft.com/en-us/library/ee259689.aspx
114
http://technet.microsoft.com/en-us/library/cc770558.aspx
Page 81 of 109
Planning Guide Appendix
Cluster Strategy
The objective of the cluster strategy is to avoid outages due to the failure of a TFS Data-tier server. You can
consider mirroring or clustering to handle failover.
Option Description Rating 115 Details Done
3 Active-Active Each node must be capable of handling the load of all SQL Server
Cluster instances during a SQL Server cluster node failover.
+ Use each node to run one or more SQL Server instances.
Split TFS and SharePoint data onto separate active node servers, with one
passive standby server. Read here 118 for more information.
115
Unofficial failover rating, to indicate that clustering is preferred over mirroring
116
http://msdn.microsoft.com/en-us/library/aa980644(v=vs.90).aspx
117
http://msdn.microsoft.com/en-us/library/aa980644(v=vs.90).aspx
118
http://msdn.microsoft.com/en-us/library/aa980644(v=vs.90).aspx
119
http://msdn.microsoft.com/en-us/library/aa980644(v=vs.90).aspx
120
http://msdn.microsoft.com/en-us/library/ff877884.aspx
Page 82 of 109
Planning Guide Appendix
Farms Learning
DNS Aliases
Use as many DNS alias names as possible. This lets you scale in new systems while keeping the original
system online. After the new system is prepared, you can transparently switch to the new server without
downtime, or with as little downtime as possible.
If a proxy or a TFS DT fails you can simply point the DNS alias to the new system and off you go again.
The same is true for backend systems like the warehouse or analysis DB.
Example Infrastructure
The following diagram shows a possible DR environment. Although these are important part of your disaster
recovery plan, note that SQL Reporting and Analysis services, Test and Build servers, and shared network storage
failover are not shown in the diagram.
Mirroring SQL Server data might be useful in this scenario to mirror the data to a different physical location, for
example, another physical datacenter.
Page 83 of 109
Planning Guide Appendix
Walkthrough: DR Recovery - Complete Failure (aka Datacenter fire or meteor strike)
Walkthrough Checklist Single Server installation
Step Description Reference Link(s) Done
3 If you have a secondary site in place, follow the walkthrough on page 87.
4 If you dont have a secondary site in place, install the new server. See Restore a Single-Server
The guide includes: Deployment to New
Hardware (2010) 121.
Prepare the new hardware.
Restore the databases.
See Restore a Single-Server
Install and configure TFS.
Deployment to New
Reconnect services and users. Hardware (2012) 122.
5 Make sure that the DNS entry points to the new TFS server. If no DNS See Configure TFS to use
entry exists, you need to inform the TFS users of the new TFS location. FQDN 123.
Table 45 Walkthrough: Single Server installation
3 If you have a secondary site in place, follow the walkthrough on page 87.
4 If you dont have a secondary site in place, start recovering all servers, by
following = the walkthrough on page 85.
6 If you are using SharePoint servers, recover them by following the See Backup and recovery for
walkthrough on page 95. SharePoint Server 2010 124.
See Backup and recovery for
SharePoint Server 2013 125.
See Protecting and restoring
the farm (Windows SharePoint
Services 3.0) 126.
121
http://msdn.microsoft.com/en-us/library/ff459215.aspx
122
http://msdn.microsoft.com/en-us/library/hh529826.aspx
123
http://msdn.microsoft.com/en-us/library/cc752948(v=VS.90).aspx
124
http://technet.microsoft.com/en-us/library/ee662536(v=office.14).aspx
125
http://technet.microsoft.com/en-us/library/ee662536(v=office.15).aspx
126
http://technet.microsoft.com/en-us/library/cc287741(office.12).aspx
Page 84 of 109
Planning Guide Appendix
Step Description Details Done
that indicate data corruption, hardware related error messages on the hosting server, or an inability to connect to
any of the TFS databases.
3 Install and configure TFS See Install and configure Team Foundation Server129.
2 Stop services. See Stop Services that Team Foundation Server Uses132.
6 Restart TFS services. See Restart Services that Team Foundation Server Uses136.
127
http://msdn.microsoft.com/en-us/library/hh529825.aspx
128
http://msdn.microsoft.com/en-us/library/hh529829.aspx
129
http://msdn.microsoft.com/en-us/library/hh529828.aspx
130
http://msdn.microsoft.com/en-us/library/hh529824.aspx
131
http://msdn.microsoft.com/en-us/library/ms252458.aspx#RequiredPermissions
132
http://msdn.microsoft.com/en-us/library/ms252458.aspx#StopServices
133
http://msdn.microsoft.com/en-us/library/ms252458.aspx#RestoreDatabases
134
http://msdn.microsoft.com/en-us/library/ms252458.aspx#UpdateAccounts
135
http://msdn.microsoft.com/en-us/library/ms252458.aspx#RebuildDataWarehouse
136
http://msdn.microsoft.com/en-us/library/ms252458.aspx#RestartServicesForTFS
Page 85 of 109
Planning Guide Appendix
Step Description Reference Link(s) Done
2 Install and configure SQL Server. See Install and Configure SQL Server on new hardware139.
3 Stop services. See Stop Services that Team Foundation Server Uses140.
6 Modify Reporting Service configuration. See Change the Database in Reporting Services
Configuration Manager143.
7 Prepare SQL Server. See Prepare the New SQL Server or Instance for Team
Foundation Server144.
8 Modify the database owner. See Change the Ownership of the Restored Databases145.
9 Redirect TFS to restored Project See Redirect Team Foundation Server to Remote Collection
Collections. Databases146.
11 Register restored databases. See Register the Location of the Restored Databases148.
13 Clear the Application tier server Cache. See Clear Data Cache on Server150.
14 Restart TFS services. See Restart Services that Team Foundation Server Uses151..
137
http://msdn.microsoft.com/en-us/library/ms252458.aspx#RefreshDataCache
138
http://msdn.microsoft.com/en-us/library/ms252516.aspx#RequiredPermissions
139
http://msdn.microsoft.com/en-us/library/ms252516.aspx#InstallAndConfigure
140
http://msdn.microsoft.com/en-us/library/ms252516.aspx#StopServices
141
http://msdn.microsoft.com/en-us/library/ms252516.aspx#RestoreDB
142
http://msdn.microsoft.com/en-us/library/ms252516.aspx#RedirectSPT
143
http://msdn.microsoft.com/en-us/library/ms252516.aspx#ChangeSQLRS
144
http://msdn.microsoft.com/en-us/library/ms252516.aspx#ConfigNewSQL
145
http://msdn.microsoft.com/en-us/library/ms252516.aspx#ChangeOwnership
146
http://msdn.microsoft.com/en-us/library/ms252516.aspx#RedirectSQLRTPC
147
http://msdn.microsoft.com/en-us/library/ms252516.aspx#UpdateNetworkService
148
http://msdn.microsoft.com/en-us/library/ms252516.aspx#RegisterDB
149
http://msdn.microsoft.com/en-us/library/ms252516.aspx#RestoreWarehouse
150
http://msdn.microsoft.com/en-us/library/ms252516.aspx#ClearData
151
http://msdn.microsoft.com/en-us/library/ms252516.aspx#RestartServices
152
http://msdn.microsoft.com/en-us/library/ms252516.aspx#RefreshDataCache
Page 86 of 109
Planning Guide Appendix
Walkthrough: DR Recovery - AT Failure
Sometimes its easier to switch to a new Application tier than to fix a corrupt Application tier.
NOTE
Implementing a new Application-tier server is recommended in case of hardware failures on disks or software
issues like viruses. Because the Application tier holds just temporary cache data files, you dont lose data, with
the exception of log files, when you start over on a new system.
It is recommended to always know where the media to rebuild the Application Tier is. This reduces the time of
getting your environment back up and running. Having the pressure of wheres the media is the something you
should take out of the equation.
The following walkthrough summarizes the implementation of a new Application tier (AT) server. Its assumed
that the server is a dedicated AT server and that it is not hosting other servers such as Reporting or Build
services.
Step Description Reference Link(s) Done
1 Verify the system requirements and server considerations. See System Requirements for Team
Foundation Server 153.
3 Update the Team Foundation Application tier to pre-disaster See Latest Updates for Team Foundation
patch level. Server 2012 155.
5 If a Network Load Balancing solution is used, reconfigure it to How to: Create a Team Foundation
include the new TFS AT. Server Farm (High Availability) 156.
6 Alternatively, update the DNS entry for TFS to direct users to Configure TFS to use FQDN 157.
the new TFS AT location. Configuring NLB for TFS 2010 158.
153
http://msdn.microsoft.com/en-us/library/vstudio/dd578592.aspx
154
http://msdn.microsoft.com/en-us/library/dd793167.aspx
155
http://social.msdn.microsoft.com/Forums/en-US/tfsadmin/thread/33034618-778d-423c-9cca-1b4b6edd71fd
156
http://msdn.microsoft.com/en-us/library/ee259689.aspx
157
http://msdn.microsoft.com/en-us/library/cc752948(v=VS.90).aspx
158
http://blogs.msdn.com/b/tfssetup/archive/2010/11/12/configuring-nlb-for-tfs-2010.aspx
Page 87 of 109
Planning Guide Appendix
Is the secondary environment completely referenceable with different URLs?
How will the data from the primary environment be transferred and incorporated into the secondary
environment?
How will the primary environment be updated with the secondary environments updates when you
switch back to the primary environment?
Can the secondary environment be validated and tested while the primary environment is available?
Is patching of the secondary environment co-ordinated with patches that have been applied to the
primary environment?
How will use of the secondary environment be communicated and then how will you revert back to the
primary environment?
How will you manage dependent components like build and proxy servers outside of the primary sites
environment?
You should determine the amount of effort from all parties involved for what it will take to fail over and use this
in identifying the criteria to make that call in going to the secondary site.
Review the business requirements for going to the secondary site.
Determine all necessary participants (database, development, end users, infrastructure and testers) and
have confirmation of availability if a failover is required.
Determine the time required to prepare the secondary site. Depending on the location and timeliness of
getting the primary sites data, this could be a time-consuming task itself.
Test the move to the secondary site and track the time.
Failing over to secondary site is not a trivial task, so documenting, planning, and testing are key to having a high
level of confidence. Ownership of tasks should be established you should test your plan at least once a year.
Walkthrough Checklist
With this walkthrough, these are the conditions:
An extreme condition has been met and the primary site is going to be unavailable for at least a month.
The primary and secondary sites are in the same domain. This ensures that the domain remains active.
The primary site was a single application tier and the secondary site is a single application tier.
Both the primary site and the secondary site have a warehouse and an analysis database data tier.
The secondary site is enabled from the last failover test.
The primary sites TFS databases are on disk images. These images must be ready-to-use TFS databases
and be available to the secondary site within four hours.
Proxy and build servers exist outside of the primary site. The secondary site doesnt have build or proxy
servers.
SharePoint is used for project portals.
Reporting Services is used for reporting.
The same patches have been applied to both the primary and the secondary sites.
All service accounts are domain accounts and are the same for both the primary and secondary sites.
The participants in the failover process know what activities theyve been assigned.
Activities are documented and tasks are managed by a central person. This person tracks all the tasks
that must be completed before the secondary site is pronounced ready to use.
Step Description Details Done
1 Determine Point in Time Identify when the data images are from and initiate the process of making
of Database Images them available to the secondary sites database environment.
2 Communicate Failing Alert teams that the primary site is unavailable and that any changes after the
Over to Secondary Site point in time from step 1 will have to be reapplied in the secondary site.
Page 88 of 109
Planning Guide Appendix
Step Description Details Done
3 Prepare Secondary Site Stop the secondary sites TFS related services:
SharePoint timer service via the Services interface.
Reporting Services service via the Services interface.
Warehouse processing via the TFS Administration console.
Quiesce TFS from an elevated command prompt using the
TFSServiceControl quiesce command.
Detach TFS_Configuration, Team Project Collection database and SharePoint
content databases.
4 Attach Data Images to Attach TFS_Configuration, Team Project Collection database and SharePoint
Secondary Site content databases that have been presented as a different drive letter than
what was used before.
Note: The TFS Warehouse and Analysis Services databases will be rebuilt and
do not need to be attached.
6 Validate Connections To validate that TFS databases are properly mapped, you can query the
configuration database to see the connection strings that TFS will use. Run
this query: SELECT * FROM Tfs_Configuration.dbo.tbl_ServiceHost
8 Review Logs Check AT and DT Windows and SQL logs for messages.
9 Validate Secondary Site Designated development tester will confirm that secondary URI is accessible
Connectivity in Visual Studio.
10 Update Build and Proxy In the Build Servers TFS Admin Console, change the connection to the
Server to Use Secondary secondary sites URI. In the Proxy Servers TFS Admin Console, change the
Site connection to the secondary sites URI.
11 Validate Build and Proxy Designated development tester will confirm that reconfigured Build and
Server Usability Proxy servers are usable.
12 Communicate Availability Gradually inform teams that the secondary site is available.
of Secondary Site
13 Monitor Secondary Site Designated TFS administrator will monitor event logs for connections that
refer back to the primary site.
Table 49 Walkthrough: Failingover to Secondary Site
Page 89 of 109
Planning Guide Appendix
Step Description Details Done
1 Determine Point in Time of Identify when you want to take the data images from the secondary site
Database Images and make them available to the primary sites database environment.
2 Communicate Plan to Return Co-ordinate an appropriate time with teams for bringing the primary site
to Primary Site back on-line.
3 Stop the Secondary Site Stop the secondary sites TFS related services:
SharePoint timer service via the Services interface.
Reporting Services service via the Services interface.
Warehouse processing via the TFS Administration console.
Quiesce TFS from an elevated command prompt using the
TFSServiceControl quiesce command.
Detach TFS_Configuration, Team Project Collection database and
SharePoint content databases.
4 Prepare Primary Site Stop the primary sites TFS related services:
SharePoint timer service via the Services interface.
Reporting Services service via the Services interface.
Warehouse processing via the TFS Administration console.
Quiesce TFS from an elevated command prompt.
Detach TFS_Configuration, Team Project Collection database and
SharePoint content databases.
Bring the disk images back and make them available to the primary sites
database server.
5 Attach Data Images to Attach TFS_Configuration, Team Project Collection database and
Primary Site SharePoint content databases that have been presented as a different
drive letter than what was used before.
Note: The TFS Warehouse and Analysis Services databases will be rebuilt
and do not need to be attached.
7 Validate Connections To validate that TFS databases are properly mapped, you can query the
configuration database to see the connection strings that TFS will use.
Run this query:
SELECT * FROM Tfs_Configuration.dbo.tbl_ServiceHost
9 Review Logs Check AT and DT Windows and SQL logs for messages,
10 Validate Primary Site Designated development tester will confirm that secondary URI is
Connectivity accessible in Visual Studio.
Page 90 of 109
Planning Guide Appendix
Step Description Details Done
11 Update Build and Proxy In the Build Servers TFS Admin Console, change the connection to the
Server to Use Primary Site primary sites URI.
In the Proxy Servers TFS Admin Console, change the connection to the
primary sites URI.
12 Validate Build and Proxy Designated development tester will confirm that reconfigured Build and
Server Usability Proxy servers are usable.
13 Communicate Availability of Gradually inform teams that the primary site is available.
Primary Site
14 Monitor Primary Site Designated TFS administrator will monitor event logs for connections that
refer back to the secondary site.
Table 2 Walkthrough: Going Back to Primary Site
Service Confirm the password still works and the proxy service account is not TFS Expert,
Account disabled. If it is disabled, enable it. If password has changed, reconfigure Active Directory
the proxy and retype the password. Administrator
Hard Disk Check if hard disk usage is below 10% of available disk space on the TFS Expert,
drive. Infrastructure
On virtual machine: increase disk. Administration
On physical: mount bigger disk, reconfigure proxy to use the new
partition.
159
http://msdn.microsoft.com/en-us/library/vstudio/dd578644.aspx
Page 91 of 109
Planning Guide Appendix
Area Solution Involved Checked
personas
Antivirus Is there antivirus software installed on the proxy? Some antivirus tools TFS Expert,
block access to suspect files. Check the antivirus log file for recent virus Security
activities. Administrator
Developers Developers report receiving a proxy server error message in the Visual TFS Expert,
Studio output window that they cannot connect to the proxy server. Infrastructure
Determine if the proxy server is accessible remotely via the proxy Admin
statistics URL.
Table 50 Walkthrough: Investigating Proxy failures
1 Verify the system requirements and server See How to: Install Team Foundation Proxy
considerations. and Set Up a Remote Site 160
See Proxy Server Considerations, page 21.
160
http://msdn.microsoft.com/en-us/library/vstudio/ee248710.aspx
Page 92 of 109
Planning Guide Appendix
DR Investigation - Walkthrough
Area Solution Involved personas Checked
End user Users reports that after checking in code from Visual Studio the build fails TFS Expert,
Visual intermittently. End user
Studio Check if build server is running. It could be down.
If you can log in to the server, check if the TFS Admin Console
accessible on the server. The log should be reviewed for changes
that adversely affected the server.
Check if the Build Controller is running,
Check if the Build Agent is running,
Service Confirm that the password still works. If it is disabled, enable it. If password TFS Expert,
Account has changed, reconfigure build and retype the password. Active Directory
Administrator
Enable Enable logging on the build sever to get additional diagnostics information. TFS Expert
Logging Create a config file with the name TFSBuildServiceHost.exe.config and paste
the following configuration information in it:
<configuration>
<system.diagnostics>
<switches>
<add name="BuildServiceTraceLevel" value="4"/>
</switches>
<trace autoflush="true" indentsize="4">
<listeners>
<add name="myListener"
type="Microsoft.TeamFoundation.TeamFoundationTextWriterTraceListener,M
icrosoft.TeamFoundation.Common, Version=10.0.0.0, Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a" initializeData="C:\TFSBuildService
Logs\TFSBuildServiceHost.exe.log" />
<remove name="Default" />
</listeners>
</trace>
</system.diagnostics>
</configuration>
For TFS 2012 Build Server, drop the file in this location (if you installed to the
default location): C:\Program Files\Microsoft Team Foundation Server
11.0\Tools
Antivirus Is there antivirus software installed on the proxy? Some antivirus tools block TFS Expert,
access to suspect files. Check the antivirus log file for recent virus activities. Security
Administrator
Services Builds wont run. Check if Remote Connections Access Manager service is TFS Expert,
running. Company policies might not allow the Telephony service to run. This Security
Administrator,
Page 93 of 109
Planning Guide Appendix
Area Solution Involved personas Checked
is a problem because the RCMA service depends on Remote Connections Infrastructure
Access Manager service. Administration
Table 52 - Walkthrough: Build failure investigation
5. Make sure that the build controller starts on the TFS build server.
7. Make sure the build agent(s) starts on the TFS build server.
10. Install the TFS build server software and configure it to connect How to: Install Team Foundation
to the TFS App Server. Build and Set Up 161
11. Run the Best Practice Analyzer (BPA). Best Practice Analyzer is part of the
Power tool. To Install Power Tools
162
161
http://msdn.microsoft.com/en-US/library/vstudio/ms181712.aspx
162
http://visualstudiogallery.msdn.microsoft.com/b1ef7eb2-e084-4cb8-9bc7-06c3bad9148f
163
http://msdn.microsoft.com/en-US/library/vstudio/ms181712.aspx
164
http://visualstudiogallery.msdn.microsoft.com/b1ef7eb2-e084-4cb8-9bc7-06c3bad9148f
Page 94 of 109
Planning Guide Appendix
Step Description Reference Link(s) Done
The attempt to upload a file that exceeds the maximum upload size will also result in an error or a timeout. This task
NOTE
isnt covered in this guide. For additional information, consult the disaster recovery strategy sources of SharePoint on
http://technet.microsoft.com/en-us/library/ff628971(v=office.14).aspx for SharePoint Server 2010 and
http://technet.microsoft.com/en-us/library/ff628971.aspx for SharePoint Server 2013.
Error Go to the SharePoint Central Administration and check for the first TFS / SharePoint
Determination error(s). Expert
SharePoint Confirm that the password still works and that the SharePoint TFS / SharePoint
Service service account is not disabled. If it is disabled, enable it. If the Expert
Account password has changed, reconfigure SharePoint and retype the Active Directory
password. Administrator
Database Connect to the SharePoint database via Microsoft SQL Server TFS Expert, SQL Server
Connection Management Console, and check the SQL server health: CPU, Administrator
and SQL memory, and disk usage.
Server
Page 95 of 109
Planning Guide Appendix
Area Solution Involved personas Checked
2013 for more information on ports and security hardening
165
of SharePoint.
If applicable, shut down the firewall or switch to a lower security
level to easily diagnose this problem.
Virus Check your antivirus solution. Some viruses can cause errors on http TFS Expert, Security
communications. Administrator
IIS Logging C:\Inetput\Logs\... <location of SharePoint IIS logging>. Review TFS/SharePoint Expert,
for SharePoint connections being made/not made by users. Do you see any status Network Administrator
connections other than 200s? Are users connecting?
SharePoint Determine if data is being logged into the SharePoint database for SharePoint Expert
Content the TPC sites
Database SELECT TOP 100 *
FROM [WSS_Content_<DB>].[dbo].[EventLog]
ORDER BY EventTime DESC
Table 55 - Walkthrough: One TFS Build Controller and multi-server TFS Build agents
165
http://technet.microsoft.com/en-us/library/cc262849.aspx
Page 96 of 109
Planning Guide Appendix
Authoring Reports
There are many sources for report information with TFS. You can use the Perfmon counters, data captured in the
TFS databases, SQL Server statistics, the TFS API, and even the Windows event logs. Using these sources gives
you a fairly complete picture of the state of your TFS environment.
1 Install the Download and install Grants reports. Depending on your environment
Administrative configuration, you might need DBA assistance. The TFS implementation could
Report Pack. have restrictions on who can perform particular Reporting Service activities such
as configuring data sources.
2 Review the Reports. Review the information available in the admin reports. Determine which ones
should be emailed to the TFS support team members on a regularly scheduled
basis.
3 Understand how to See Getting Started with Report Builder 3.0 167 for more details.
use reporting tools
like Report Builder.
4 Review how to See Create, Customize, and Manage Reports for Visual Studio ALM168 and
create customized Creating and Customizing Reports for TFS 169 more details.
reports.
5 Leverage existing The TFS Warehouse has view options to report on activity. Use these views as a
views. basis for your reports and implement proper joins.
Table 56 Walkthrough: authoring reports
1 Use SQL Server Management Server to query data. Sys Admin access is required in SQL Server.
166
http://blogs.msdn.com/b/granth/archive/2010/07/12/administrative-report-pack-for-team-foundation-server-2010.aspx
167
http://technet.microsoft.com/en-us/library/dd220460.aspx
168
http://msdn.microsoft.com/en-us/library/vstudio/dd578644.aspx
169
http://msdn.microsoft.com/en-us/library/ff647430.aspx
Page 97 of 109
Planning Guide Appendix
Step Description Details Done
3 Identify if reporting service reports are running Use the ExecuteLog2 view to view long running
long. reports. Reports that capture a lot of data can slow
done other reports that are currently running.
Table 57 Walkthrough: Reporting with SQL Server
2 Identify what you logparser.exe -i:EVT "SELECT * FROM Application WHERE TimeGenerated > '2013-
need. 01-1 00:00:00' AND EventType IN (1;2) ORDER BY TimeGenerated DESC" -o:CSV -
q:ON -stats:OFF >> c:\temp\TFSAppEvents.csv
2 Review PAL functionality. Review how data can be selected with PAL.
3 Determine the relevant Analyze the results of importing performance counter files.
information to review. What is the user patterns like?
Items to monitor
Items flagged on the report
170
http://msdn.microsoft.com/en-us/library/vstudio/dd578644.aspx
171
http://msdn.microsoft.com/en-us/library/vstudio/dd578644.aspx
Page 98 of 109
Planning Guide Appendix
Walkthrough-Create Report to Monitor Average Response Time Counters
Step1: Create Counters to monitor
To collect the counters to monitor, we will use Logman. Logman creates and manages Event Trace Session and
Performance logs and supports many functions of Performance Monitor from the command line.
The first step is to create a text file (c:\TFSCouters.txt) with all the counters you want to monitor. For the current
scenario, we will be monitoring the Average Response Time Counter.
Page 99 of 109
Planning Guide Appendix
Sample Script
'==========================================================================
' VBScript Source File -- Created with SAPIEN Technologies PrimalScript 4.0
'
' NAME: GrpNestingWeights.vbs
'
' AUTHOR: Guy Teverovsky, Microsoft
' DATE : 3/23/2009
'
' COMMENT: The script is useful for identifying "heavy" groups - those that
' when added to user will "donate" a lot of groups to user token
' through group nesting. The script will output the list of all
' security enabled groups in the domain and will tell how many
' groups each one "donates".
'
' i.e.: if GRP1 is member of GRP2 and GRP3 and GRP3 is member
' of GRP4, then membership in GRP1 "donates" 4 groups
' This will not help you to calculate user access token, but will
' give you a good estimate of over-abused groups you want to
' take care of.
'
' P.S.: As a bonus, the script will also detect nesting loops -
' if GRP1 is member of GRP2 and GRP2 is member of GRP3,
' and GRP3 is member of GRP1, you will be notified about
' the loop.
'
' COPYRIGHT:
' Copyright Microsoft Corporation. All Rights Reserved. This code
' is released under the terms of the Microsoft Public License (MS-PL,
' http://opensource.org/licenses/ms-pl.html.)
' This is sample code only, do not use in production environments!
'==========================================================================
Option Explicit
Dim objRootDSE
Dim objGroupList
Dim strBaseDn
Dim iGroupCount : iGroupCount = 0
EnumGroups
objLogFile.Close
objAdoCmd.CommandText = strLdapSearchString
iGroupWeight = GetGroupNestingWeight(objRS.Fields(0).Value)
OutputLine objRS.Fields(2).Value & vbTab & iGroupWeight, False
Function GetGroupNestingWeight(strGroupAdsPath)
Dim objGroup
Dim objMemberOf, strMemberDn
Dim iGroupWeight
iGroupWeight = 1
GetGroupNestingWeight = 1
Constraints
Note that the script:
Is not supposed to serve as an example of good programming practices.
Does not account for cross-domain loops.