Evaluation Portal
Evaluation Portal
Evaluation Portal
Evaluation Division
March 2011
Re s e a r c h a n d E v a l u a t i o n
Ci4-67/2011E-PDF
978-1-100-18900-0
Ref. No.: ER201104.01E
Table of contents
Executive summary ............................................................................................................................. iv
1. Background .............................................................................................................................. 1
1.1. Introduction ....................................................................................................................................... 1
1.2. Profile of the GTC-IP Initiative .......................................................................................................... 1
1.2.1. Policy and program context .............................................................................................................. 1
1.2.2. Initiative description and objectives .................................................................................................. 2
1.2.3. Structure and navigation of the GTC-IP initiative sites...................................................................... 3
1.2.4. Resources ........................................................................................................................................ 4
1.3. Roles and responsibilities ................................................................................................................. 4
1.3.1. Citizenship and Immigration Canada ................................................................................................ 4
1.3.2. Human Resources and Skills Development Canada ........................................................................ 6
1.3.3. The provinces and territories ............................................................................................................ 8
-i-
Appendix B: Evaluation matrix ......................................................................................................... 54
List of tables
Table 2-1: Evaluation issues and questions ............................................................................................... 10
Table 2-2: Online user survey completions by source ................................................................................ 12
Table 2-3: Intermediary survey completions by organization type .............................................................. 13
Table 2-4: Federal/provincial/territorial web analytics and log files provided .............................................. 14
Table 3-1: Examples of tools and information that have been shared among GTC-IP partners ................. 26
Table 3-2: Funds allocated and spent by CIC (2007-08 to 2009-10) .......................................................... 29
Table 3-3: Funds allocated and spent by HRSDC (2007-08 to 2009-10).................................................... 29
Table 3-4: Funds allocated and spent by provinces and territories (2007-08 to 2009-10) .......................... 30
Table 3-5: Use of the provincial/territorial portals over time (2008 to 2010)* .............................................. 34
Table 3-6: Studies Commissioned by the Provinces and Territories on their Portals ................................. 39
List of figures
Figure 1-1: The GTC-IP initiative ................................................................................................................... 3
Figure 3-1: Growth in usage of the WiC, 2008-2010 (# of unique visitors in March of each year) ............... 32
Figure 3-2: Percent of online users who said it was easy to find information ............................................... 37
Figure 3-3: Comparison of specific attributes of usability and functionality of GTC and WiC by intermediaries
.................................................................................................................................................. 37
Figure 3-4: Comparison of specific attributes of quality of GTC and WiC by intermediaries ........................ 41
Figure 3-5: Percent of online users who found specific components of the GTC-IP to be useful................. 43
Figure 3-6: Frequency of intermediaries recommending WiC or GTC ......................................................... 43
Figure 3-7: Percent of online user survey respondents indicating instances where information on GTC-IP
was useful/helpful ...................................................................................................................... 44
Figure 3-8: Percent of intermediaries who found specific aspects of the GTC-IP useful for new and
prospective immigrants .............................................................................................................. 45
- ii -
List of acronyms
CA Contribution Agreement
CAPAR Canada‟s Action Plan Against Racism
CIC Citizenship and Immigration Canada
COIA Canada Ontario Immigration Agreement
DFAIT Department of Foreign Affaires and International Trade
FCR Foreign Credential Recognition
FPT Federal and Provincial/Territorial
FQR Foreign Qualifications Recognition
GoC Government of Canada
GTC Going to Canada (referring to the Going to Canada website)
GTC-IP Going to Canada Immigration Portal
HRSDC Human Resources and Skills Development Canada
IMTB Information Management and Technology Branch (CIC)
IO Internal Order
ISAP Immigrant Settlement and Adaptation Program
LMI Labour Market Information
LOU Letter of Understanding
MIIO Municipal Immigration Information Online
MOU Memorandum of Understanding
OGD Other Government Department
OMC Operational Management Coordination Branch (CIC)
P/T Provincial and Territorial
SPO Service Provider Organization
WiC Working in Canada (website)
- iii -
Executive summary
The Going to Canada Immigration Portal initiative
The Government of Canada is committed to making the country a destination of choice for
skilled immigrant workers and foreign students. As part of this mandate, the Government seeks
to provide accurate, free, and accessible information that helps people make informed decisions
about the immigration, employment, and settlement process. The 2005 Budget allocated funding
for the development of an integrated and comprehensive Going to Canada Immigration Portal
(GTC-IP), aimed at creating an authoritative access point for seamless online content and tools.
The Going to Canada Immigration Portal (GTC-IP) or simply, “the Portal,” consists of two
interlinked websites: the Going to Canada (GTC) site (www.goingtocanada.gc.ca), operated by
Citizenship and Immigration Canada (CIC) and the Working in Canada (WiC) site
(www.workingincanada.gc.ca), run by Human Resources and Skills Development Canada
(HRSDC). As a separate entity from the CIC departmental website (www.cic.gc.ca), the GTC-IP
was envisioned as an organized gateway, helping to structure a dispersed and diverse collection of
information specific to the needs of the target audience.
As part of a wider GTC-IP Initiative, CIC also provides annual contribution funding to provinces
and territories (P/Ts), who work collaboratively with CIC and HRSDC to develop similar web
portals of their own, which are linked to the GTC-IP. The GTC-IP Initiative also brings together
many other stakeholders, such as municipalities, other government departments (OGDs) and
service provider organizations (SPOs), who share information and review different approaches to
the development of online content for potential immigrants and newcomers to Canada.
With the GTC-IP acting as a navigational hub, the Portal Initiative integrates information using
links and commonly shared tools. In this way, the Initiative seeks to facilitate the distribution of
reliable information, which helps people make more informed decisions about coming to Canada
and to better prepare them, before and after they arrive, for integration into the Canadian labour
market and society.
Key findings
The following section provides key findings regarding the relevance and performance of the
GTC-IP Initiative.
Relevance
The evaluation showed that the GTC-IP Initiative is a relevant program that addresses a
continuing and demonstrated need by providing complete, up-to-date, and accurate information
to its target audience in a more unified manner than most alternative sources. Users of the GTC-
- iv -
IP reported that the Portal contained the information they needed and that it increased their
knowledge of living and working in Canada. As well, they reported that the information on the
GTC-IP helped them make decisions about coming to Canada, prepare for the immigration
process, and integrate into Canadian society upon arrival.
Furthermore, the GTC-IP Initiative demonstrates a high degree of alignment with Federal
priorities related to attracting a skilled workforce, and the successful integration of newcomers.
The Federal government is well placed to provide national-level, authoritative information and to
coordinate the provision of more local-level information provided by provinces, territories and
municipalities.
Nevertheless, a significant amount of GTC content, which was originally developed for the
purposes of the Portal, has been copied and incorporated into the CIC main site. It was felt that
the much larger user population of the CIC main site would help to drive users to access Portal
content. However, the lack of distinction in mandate for content provision between GTC and
the CIC main site challenges the relevance of the GTC site moving forward. The evaluation
showed uncertainty regarding future directions of the GTC-IP Initiative resulting from the
pending decision by CIC to fully integrate the GTC-IP into the CIC main site. The implications
of such a move have not been formally discussed with the partners of the Initiative, who
highlighted potential issues with integration as part of this evaluation.
Performance
The GTC-IP Initiative is a relatively new endeavour and has experienced some challenges
associated with its stage of development. The Initiative has operated under a governance model
that has not been clearly defined and implemented and does not have well-articulated decision-
making processes among partners. Key informants also reported a lack of clarity in roles and
responsibilities across partners. Governance of the Initiative within CIC has been particularly
challenging as a result of a decentralized system with four branches working independently on
overlapping program components, and an historical lack of clear leadership. The internal
governance of the WiC by HRSDC, however, consists of one team working together, allowing
for responsiveness, flexibility, creativity and innovation in WiC‟s design and delivery.
The fragmented nature of CIC‟s governance structure may have contributed to inconsistent
tracking of allocations and expenditures, including to P/Ts and across Branches. A review of
documentation also revealed that provincial and territorial reporting on expenditures was not
always completed in a timely fashion and differed in level of detail across P/Ts.
Despite these early challenges, recent improvements in the delivery of the GTC-IP were noted in
the evaluation, including the identification of e-Communications Branch as the CIC Portal lead,
the creation of a series of draft documents that clarify the governance of the GTC-IP, a revised
funding model within CIC that is more responsive to the needs of the Initiative, and
improvements in timeliness of reporting by provinces and territories. As well, almost all of the
P/T websites are now fully functional. It is anticipated that these improvements will continue to
add to the performance of the GTC-IP Initiative moving forward.
The primary strength of the GTC-IP Initiative has been the development of federal/provincial/
territorial/municipal partnerships in the creation and provision of a full spectrum of online
information, from the local to the national level. Focus group participants identified this range of
information as necessary to assist their decision-making and facilitate their integration once in
Canada. The multilateral partnerships that have been created as a result of the GTC-IP Initiative
-v-
were identified as a key positive unexpected outcome. These multilateral partnerships have
resulted in the sharing of content and tools, which has created cost savings for all partners and
improved the overall quality and comprehensiveness of information available on partner sites.
The GTC-IP communications structures supporting these partnerships are effective and the bi-
annual workshops, which facilitate information sharing and relationship-building, were
highlighted as a best practice in the evaluation.
Multiple lines of evidence showed that the GTC, WiC, and P/T sites are functional, user-friendly,
relevant, easy-to-navigate, and provide high quality information. As well, these sites are providing
consistent information on immigrating to and working in Canada, likely resulting from the
extensive collaboration of Portal partners. A number of provincial/territorial portals have been
the recipient of awards and received recognition for the quality of their websites. As well, the
WiC tool has shown such success that it has been made Canada‟s authoritative source for labour
market information.
By comparison, the GTC site provides information for potential immigrants and newcomers
covering a wide range of topics. The evaluation showed that users of GTC are most frequently
accessing the site‟s information on immigrating to Canada and are less likely to be accessing
information on the site related to settling, studying, or visiting Canada‟s provinces and territories.
Provincial and territorial portals provide high quality information on settling and studying and
may be in the best position to provide this more local level information to the target audience.
The evaluation showed that more could be done to increase the awareness of the information
contained on the Portal. Promotional activities by Portal partners varied, with the promotion of
the GTC found to be limited, likely as a result of uncertainty over the permanent location of the
GTC web assets. This resulted in a lack of growth, but steady usage over time. HRSDC‟s
promotion of the WiC has been effective, facilitating growth in the profile and usage of the WiC
site over the last three years. The WiC site has also benefited from the development of
partnerships within the Initiative and the placement of the WiC tool on a number of partner sites.
A number of provinces and territories have also engaged in promotion of their sites; however,
there is little data available (e.g., usage trends around the time of promotional campaigns) to
demonstrate the effectiveness of those campaigns, or the usage of these sites overall. A wide
range of practices in web data collection are currently employed among all partners of the GTC-
IP (including provinces and territories), which limits the assessment of ongoing performance of
the Initiative.
- vi -
the authoritative source for labour market information in Canada and HRSDC should continue
its promotion and related partnership development.
Recommendations
(CIC) Develop a strategy to address the issues associated with the location of the GTC-IP
and its related content and tools, including:
The implications if full integration of the GTC-IP into the CIC main site takes place. This
could include issues concerning navigation, organization of GTC-IP components, and
content updates, for example;
The elimination of information overlap that exists between the CIC main site and the
GTC site;
The determination of which GTC content areas are appropriate to remain on this
federally operated website.
(CIC and HRSDC) Consider ways to improve the promotion and usability of GTC-IP related
content and tools.
The multilateral partnerships established through the GTC-IP Initiative were integral in allowing
for the creation and provision of consistent, high-quality information in a cost-efficient manner.
The partnerships were identified as the key strength of this Initiative and should be maintained.
The partners of the GTC-IP should continue to develop and provide users with high quality
content and tools as they continue to have a high degree of relevance. The communication
structures used by Portal partners are also effective and should be continued, particularly the bi-
annual workshops, which were identified as a best practice. Additional multilateral
communications may improve issues with governance and reporting that were identified in the
evaluation.
Recommendations
(CIC and HRSDC) Establish a plan to clarify the horizontal governance structure of the
GTC-IP, including decision-making protocols and oversight structure.
(CIC) Develop a strategy to address issues concerning CIC internal governance.
(CIC and HRSDC) Consider ways for Portal partners to further facilitate multilateral
communications.
Improvements to Portal governance and communications structures would also assist in the
development of common performance measures across the Initiative, improving partners’ ability
to assess ongoing performance. Clarity in decision-making processes and roles and
responsibilities of partners related to CIC‟s internal financial tracking system and reporting by
CIC and P/Ts would lead to benefits for the department.
Recommendations
(CIC and HRSDC) Address issues related to reporting and financial tracking:
a) (CIC) Develop a strategy to address reporting issues, both internally within CIC and
with provincial and territorial partners. This includes issues dealing with financial
- vii -
tracking, the timeliness of reporting and the clarity of reporting requirements, as well as
clarity concerning the roles and responsibilities of these partners.
b) (CIC and HRSDC) Work together to establish common performance measures and
consistent web data collection across all partners of the Portal Initiative.
- viii -
Management response
Implement-
Key Finding Response Action Accountability
ation date
CIC
The implications if full CIC has developed a strategy to move the GTC-IP The strategy will be implemented by Q1 CIC Q1 2011-12
integration of the GTC-IP content and tools to the CIC website to reduce 2011-12 resulting in Portal assets relocating (Communications)
into the CIC main site duplication and overlap, working closely with to the newly created Newcomers section of
takes place. This could HRSDC to outline the particulars of full the CIC website.
include issues concerning integration. HRSDC concurs with the approach. CIC (Integration)
navigation, organization Content reviews will be conducted on an Q4 2011-2012
of GTC-IP components Provincial and territorial partners have been annual basis to maintain its currency. and Q4 2012-
consulted regarding the new location of GTC-IP 2013
and content updates, for
example; content.
The new navigation and organization of content
was tested with newcomers and potential
immigrants through usability testing.
The elimination of CIC has completed a mapping exercise of the Based on the mapping exercise, duplication Integration Q3 2010-11
information overlap that GTC content in comparison with the settlement of content will be eliminated with the
exists between the CIC information on the CIC site to ensure that there launch of the Newcomers section of the CIC
main site and the GTC will be no duplication on the CIC site once GTC site.
site; content is integrated.
The determination of Consultations were undertaken with newcomers The appropriateness of settlement content CIC (Integration) / Q4 2011-2012
which GTC content areas in order to determine their information needs. will be based on periodic expert meetings, Communications) and Q4 2012-
are appropriate to remain The new settlement information (SIRE) takes into literature reviews and consultations with 2013
on this federally operated account these interests. specialized content providers.
website.
The SIRE information is relevant as well as The CIC site will link to local and regional
aligned with information provided by other information available on provincial and
departments in the Federal government. territorial partner websites.
- ix -
Implement-
Key Finding Response Action Accountability
ation date
Consider ways to improve CIC and HRSDC agree with this recommendation.
the promotion and
usability of GTC-IP
CIC and HRSDC will work closely to explore ways CIC and HRSDC will develop and CIC Q4 2011-12
related content and tools.
of improving the initiative‟s promotion of its web implement a plan to promote GTC-IP (Communications)
assets. This may be accomplished via the Web assets. Activities such as Google + HRSDC/ Skills &
following: Adwords campaigns will produce reports. Employment
Google Adwords promotion of HRSDC and CIC A more robust linking strategy will be
Working in Canada
Portal Assets. developed with provincial and territorial
(WiC)
partners.
The addition of CIC and HRSDC tools on
overseas mission websites.
Social Media cross-promotion of Portal assets – CIC (Integration)
HRSDC will promote CIC Portal Tools (e.g.,
Come to Canada Tool) on the social media
platforms Facebook, Twitter and YouTube.
CIC will produce new information videos on
settling in Canada.
Working in Canada (WiC) is currently being Working in Canada will continue to be HRSDC/ Skills & Q4 2012-13
promoted via the Better Jobs advertising promoted until the end of the campaign Employment Branch
campaign. This is a two-year campaign that in 2013.
launched January 10, 2011 with a focus on TV Working in Canada
and internet. (WiC)
-x-
Implement-
Key Finding Response Action Accountability
ation date
Establish a plan to clarify CIC and HRSDC agree with this recommendation.
the horizontal governance
structure of the GTC-IP,
A Letter of Understanding between CIC and The Letter of Understanding between CIC Q2 2011-12
including decision-making
HRSDC, detailing the roles and responsibilities CIC and HRSDC will be signed. (Communications)
protocols and oversight
of each department in the GTC-IP initiative, + HRSDC/ Skills &
structure.
has been drafted. Employment
Working in Canada
(WiC)
The GTC-IP Steering Committee has existing Terms of Reference for all committees CIC Q2 2011-12
Terms of Reference as outlined in the GTC-IP referenced in the Letter of (Communications)
Project Charter. Terms of Reference for other Understanding will be implemented. + HRSDC / Skills &
GTC-IP governing bodies (as outlined in the Employment
Letter of Understanding – Interdepartmental
Oversight Committee, FPT Multi-lateral Working in Canada
Steering Committee, Ad-hoc committees) (WiC)
between CIC Branches) will be developed.
CIC
Terms of Reference have been drafted for The CIC Web governance is being CIC Q3 2011-12
governance committees. developed. This governance structure (Communications)
will take into account the GTC-IP
governance to avoid duplication.
- xi -
Implement-
Key Finding Response Action Accountability
ation date
Consider ways for Portal CIC and HRSDC agree with this recommendation. CIC will implement quarterly CIC Q1 2011-12
partners to further teleconference calls with (Communications,
facilitate multilateral provincial/territorial partners for the Integration)
communications. planning and monitoring cycles
throughout the fiscal year.
Develop a strategy to CIC agrees with this recommendation. Internally, CIC Communications will CIC Q1 2011-12
address reporting issues, receive all funding at the beginning of a (Communications)
both internally within CIC fiscal year and distribute allocations to
and with provincial and each CIC branch involved in the GTC-IP.
territorial partners. This Additionally, each branch involved will
includes issues dealing be asked to use the existing Internal
with financial tracking, Order (IO) for the GTC-IP and provide to
the timeliness of CIC Communications Branch a quarterly
reporting and the clarity report on expenditures.
of reporting
requirements, as well as With respect to the reporting CIC (Integration Q2 2010-11
clarity concerning the requirements in the contribution Program
roles and responsibilities agreements with provinces and Management)
of these partners. territories, CIC will develop a strategy to
address issues of timeliness, consistency
in level of detail in action plans and
progress reports by clarifying reporting
requirements via more detailed
instructions.
- xii -
Implement-
Key Finding Response Action Accountability
ation date
- xiii -
1. Background
1.1. Introduction
This report presents the results of an evaluation of the Going to Canada Immigration Portal
(GTC-IP) Initiative. Guided by an evaluation matrix1 and logic model (See Appendix A1 & A2),
the study examined the Initiative‟s relevance and performance, and sought to provide key
recommendations which can facilitate ongoing improvements to the operation of the GTC-IP
and future policy decisions. The services of Malatest and Associates consulting firm were retained
by Citizenship and Immigration Canada (CIC) to assist with the conduct of the evaluation
between April and December, 2010.
The report is organized into four sections:
Section 1 contains background information about the GTC-IP Initiative;
Section 2 provides information on the evaluation framework, methodologies, and
limitations;
Section 3 presents the evaluation findings by themes of relevance and performance; and
Section 4 presents overall conclusions and recommendations.
1 Results-Based Management and Accountability Framework for the Going-to-Canada Immigration Portal (GTC-IP),
CIC and HRSDC, May 2009.
2 Human Resources and Social Development Canada, Reports on Plans and Priorities, 2007-2008.
3 Citizenship and Immigration Canada, Reports on Plans and Priorities, 2007-2008.
4 Canadian Social Trends 2007, Immigrants‟ perspectives on their first four year in Canada: Highlights from three waves
1
Government of Canada would ensure more successful integration of new immigrants into the
Canadian economy and into communities, and that it would implement measures to inform
prospective immigrants and encourage them to acquire necessary credentials before they arrive in
Canada. Subsequently, in Budget 2005, the Government of Canada made a commitment to invest
$100 million over five years towards an integrated client service delivery strategy within CIC,
which included the development of an Immigration Portal. The Portal was intended to address
the identified lack of specific and relevant information available to prospective and recent
immigrants.
The GTC-IP is also linked to the Government of Canada‟s Internationally Trained Workers
Initiative and Advantage Canada commitments to improve labour market efficiency and increase
the quantity and quality of the Canadian workforce (reiterated in the October, 2007 Speech from
the Throne).
2
4. Help immigrants make local connections and informed decisions about where to settle
through links with provinces, territories, municipalities and communities; and,
5. Build effective long-term relationships with stakeholders and partners.
6 While NWT and NU have Contribution Agreements, they do not currently have a Portal and there is no evidence
of NU receiving contribution funding over the period covered by the evaluation.
7 In addition to developing their own provincial and territorial portals, four of the provinces have provided funding
and/or support for the development of municipal portals. The municipal portals in Ontario are funded through the
Contribution Agreement with Ontario.
3
1.2.4. Resources
The GTC-IP Initiative is a federally funded horizontal initiative. Costs for the GTC-IP Initiative
over 5 years (2005-06 to 2009-10) amount to $47.5 million, with $9.4 million provided to
HRSDC and $38.1 million provided to CIC, of which $27.8 million was then provided to
provinces and territories for the development of P/T portals, with the exception of Ontario
where funding was used to support the development of municipal portals. The ongoing annual
budget for the GTC-IP is $8.1 million for 2010-11 onward: $1.0 million for HRSDC and $7.1
million for CIC, of which approximately $5.7 million is to be allocated to the provinces and
territories.8
The following sections provide a brief description of each of the key partners in the GTC-IP
Initiative as well as their related roles, responsibilities and corresponding components of each
partner.
8Results-Based Management and Accountability Framework for the Going-to-Canada Immigration Portal (GTC-IP),
CIC and HRSDC, May 2009.
4
The Going to Canada website
The GTC website contains four content categories: information on welcoming newcomers to
Canada, immigrating to Canada, visiting Canada, and studying in Canada, as well as links to the
Working in Canada website and 10 P/T portals. Along with its French language counterpart, Se
Rendre au Canada (www.serendreaucanada.gc.ca), the GTC site also contains a number of tools,
described below.
The entry requirements tool
The entry requirements tool is a user-centric tool to help users determine requirements to
immigrate, work, study, or visit Canada. Users answer a series of questions specific to their
situation and are provided with a results page based on their answers. A step-by-step approach
simplifies information for users.
Service Provider Organizations (SPO) tool
The SPO tool helps newcomers find organizations in their area that can aid them in their
settlement. It also helps Canadians to get involved in welcoming newcomers to their community.
CIC developed this interactive tool in an effort to help new immigrants find programs and
services available in their community more quickly and easily. It allows site visitors to
anonymously answer a few short questions to confirm their location, client type and services of
interest. A given site visitor‟s responses are then used to generate a personalized list of all SPOs
that satisfies their specified criteria.
9Elements pertaining to full integration of the GTC and CIC main sites were identified in the Results-Based
Management and Accountability Framework for the Going-to-Canada Immigration Portal (GTC-IP), CIC and
HRSDC, May 2009.
5
Maintaining a dedicated labour market section managed by HRSDC. This section would use
the URL: WorkinginCanada.gc.ca; and
Reducing labour market content duplication between the CIC and Working in Canada
websites.
Due to this staged approach, there is currently a significant level of duplication and overlap in
content between the CIC and GTC sites. For example, both the CIC main site and the GTC site
hold information on settlement- and immigration-related issues. They both link to the WiC site
and the WiC tool. They both link to other external sites that hold information on housing,
transportation, health care, etc. They both house CIC tools originally designed exclusively for the
GTC site. The CIC main site, however, also contains departmental information, research
materials, publications, and other content not found on the GTC-IP, such as information related
to refugees, the citizenship process, proactive disclosures, and the application process to work in
and immigrate to Canada (e.g., procedures, forms, etc).
Working Together to Build a Better Labour Market Information System for Canada: Final Report, Advisory Panel on Labour
10
6
expanded in scope from merely providing labour market information to new and prospective
immigrants, to being seen as Canada‟ primary source of labour market information.11
The WiC website provides both static and dynamic content to its users. Static content includes
information for job seekers, newcomers, youth and students, older workers, and Aboriginal
people. Dynamic content includes the WiC tool (launched in May 2007 and described below)
which allows job seekers to select an occupation and a region (e.g., Welder and Related Machine
Operator in Saskatchewan) of Canada to produce an assortment of information, including
information on license and certification requirements, job opportunities, wages, outlook and
prospects, and skills and requirements.
11 Ibid
12 Ibid
13 A depiction of the sources of data and information provided by the WiC tool is provided in Appendix B.
14 This terms describes the ability of the WiC tool to take on the look and feel of the website in which the tool has
been embedded.
7
(e.g., documents, presentations), read and comment on the resources that have been posted, and
stay up-to-date on the latest projects or project ideas.15
A number of provinces and territories have developed and launched their own provincial or
territorial portals: British Columbia, Alberta, Saskatchewan, Manitoba, Ontario, New Brunswick,
Prince Edward Island, Nova Scotia, Newfoundland and Labrador, and Yukon. Provinces and
territories receive funding for the development of
their portals through contribution agreements with Provincial and territorial portals with their
dates of launch:
CIC. The Northwest Territories and Nunavut also
British Columbia April, 2008
have (as of March 2010) contribution agreements
with CIC but have not yet launched their portals. Alberta November, 2007
Ontario, British Columbia, and Manitoba have Saskatchewan April, 2010
separate funding arrangements for their portals. Manitoba April, 2005
Ontario‟s provincial portal is funded by the
Ontario March, 2006
Ontario government, and the contribution
agreement with CIC provides funding for the New Brunswick November, 2009
province‟s Municipal Immigration Information Prince Edward Island August, 2007
Online (MIIO) program (for more information on Nova Scotia March, 2007
the MIIO program, please see Appendix B). B.C. Newfoundland and Labrador April, 2009
and Manitoba portals receive funding through the
Yukon April, 2008
provisions of the Settlement Services Annexes to
their larger Immigration Agreements.
These P/T portals are at various stages of development with some having been fully operational
for a number of years, while others have only just launched. The launch dates of each P/T portal
is shown below (for a full list of P/T portals and their URLs please consult Appendix C).
In addition to developing their own provincial and territorial portals, four of the provinces
(Alberta, British Columbia, Saskatchewan, and Newfoundland and Labrador) have provided
funding and/or support for the development of municipal portals. The municipal portals were
not included in the scope of this evaluation.
8
2. Evaluation framework and methodology
2.1. Evaluation objectives, scope and questions
2.1.1. Objectives
The objectives of this evaluation were to assess the:
1) Relevance of this program in terms of:
a) continued need;
b) alignment with government objectives and priorities; and
c) consistency with respect to federal role and responsibilities; and
2) Performance of this program in achieving results in terms of:
a) effectiveness with respect to the intended outcomes of the programs, with a focus on
their immediate and intermediate outcomes; and
b) efficiency and economy, addressing the design and delivery approach of the GTC-IP
program, as well as best practices in other jurisdictions, with a view to understanding
the adequacy of these approaches and practices in meeting the information needs of
potential newcomers and immigrants in order for them to work and settle in Canada.
The evaluation also examined issues of governance, collaboration and priority setting between
and among GTC-IP program partners.
2.1.2. Scope
The study was undertaken in order to meet a commitment to Treasury Board to conduct a
summative evaluation of the GTC-IP Initiative in 2009-10. The evaluation was delayed by one
year in order to first conduct a validation of the Results-Based Management and Accountability
Framework (RMAF) for the program, as well as a Data Capacity Assessment in 2009-10. As such,
the evaluation, carried out from April 2010 to March 2011, covered the period from April 2005
to April 2010 and focused on the outcomes articulated in the program logic model, which
stemmed from the RMAF validation exercise. The evaluation focused specifically on the GTC-IP
and P/T portals and did not include municipal portals.
During the conduct of the evaluation, changes in the program have also occurred, which were
not addressed in the evaluation report as they are outside the period under examination.
However, these changes may influence the actions taken in response to the findings contained
herein. Of note are further discussions between Portal partners regarding full integration of the
GTC-IP into the CIC main site as well as further articulation of governance structures and
decision-making processes across Portal partners. To the degree possible, these have been
considered in the report conclusions and recommendations.
9
Survey of intermediaries;
Focus groups with users and non-users of the GTP-IP;
An assessment of the GTC-IP and P/T sites‟ quality and consistency of information by a
subject matter expert; and
Analyses of web log files and available analytics.
A crosswalk between the evaluation questions and the methodologies used to answer them is
shown in the Evaluation Framework found in Appendix A2. That framework also identifies the
section of the report in which the information pertaining to each evaluation question can be
located.
10. Does the Portal contain and provide links to relevant, up-to-date and understandable information on:
immigrating, settling, living, visiting, and studying in Canada?
working in Canada?
11. Does the target population gain knowledge and settlement information regarding provinces, territories,
and communities from across Canada?
12. Does the target population gain knowledge of immigrating to Canada and, living and working in Canada?
13. Does the Portal contribute to the target population‟s ability to make informed immigration decisions,
prepare for the immigration process and integrate into Canada upon arrival?
14. What have been the unintended outcomes of the GTC-IP, if any?
15. Is delivery of the GTC-IP efficient and cost-effective?
10
2.2. Methodologies
2.2.1. Document review
The following types of documents were reviewed during the evaluation (see Appendix D for the
complete list of documents reviewed):
CIC/HRSDC and other government department documents related to priorities and
commitments;
GTC-IP program documents such as policies, briefing notes, financial reports, statistical
reports, public opinion and other research documents, usability testing reports, search engine
optimization reports, web strategy documents, etc.;
GTC-IP web-produced reports;
GTC-IP RMAF and Data Capacity Assessment;
Annual Service Plans of BC and MB, provincial portal contribution agreements and
financial/activity reports, Letters of Understanding (LOUs) Memoranda of Understanding
(MOUs), action plans and progress reports; and
Communication and promotional materials, including frequently asked questions, press
releases, fact sheets.
11
Of those who completed the survey, 51% (n=153) did not reside in Canada, with another 4%
saying they live in Canada only temporarily, suggesting an immigrant base of respondents to the
survey. More males (56%) than females (44%) responded to the survey, with 80% of all
respondents falling within 20-49 years of age. These characteristics also align with CIC‟s past
experience in immigration and respondent patterns. Additionally, 20% (n=60) of the surveys
were completed in French.
The online user survey was available through a link to anyone who accessed the GTC website,
WiC website, the WiC Facebook page, and WiC Twitter page. In a typical month in 2010, the
WiC site received roughly 195,000 unique visitors compared to 53,000 unique visitors who went
to the GTC site. This represents a GTC-to-WiC visitor ratio of 0.27, which is comparable to the
ratio for respondents of the user survey (0.25).
16Hamilton, M. B, (2009). Online survey response rates and times: Background and guidance for industry. Retrieved
February 12, 2011 from: www.supersurvey.com/papers/supersurvey_white_paper_response_rates.pdf
12
Table 2-3: Intermediary survey completions by organization type
Organizations Email invitations sent Survey completions
Service provider organizations (SPOs) 200 62
Foreign missions in Canada 97 11
Canadian Visa offices abroad 72 42
National Associations 4 0
Sector Councils 2 1
Other17 - 9
Total 375 12518
17 These nine respondents received the email invitation to participate but self-identified as organizations other than
the indicated categories.
18 Of the completed intermediary surveys, 80% (n=99) were completed in English and 20% (n=26) in French.
19 Recent immigrants are defined as those that have immigrated to Canada within the last 5 years.
20 Focus group participants had to have come to Canada within the last five years, were over the age of 18 years old,
13
2.2.6. Subject matter expert assessment
A subject matter expert21 on immigration and settlement content assessed the quality (relevance,
currency, and clarity) of the information on the GTC, WiC, and provincial/territorial sites and
also assessed the consistency of information on the provincial and territorial portals (see
Appendix C for the list of websites assessed) with the information on the GTC-IP. The subject
matter expert reviewed content across the following six categories including: immigrating,
settling, living, visiting, studying, and working.
Alberta 2008-09
Not available
2009-10
Saskatchewan 2008-10
Not available
2009-10
Manitoba Not available 2008-10
2007-08
Ontario Not available 2008-09
2009-10
New Brunswick 2009-10 2009-2010
Prince Edward Island August 1–5, 2010 Not available
Nova Scotia Not available Not available
Newfoundland and Labrador Not available 2009-10
Yukon 2009-10 Not required
21 The subject matter expert was Dr. Adnan Turegun, Executive Director of the Centre for International Migration
and Settlement Studies (CIMSS) at Carleton University. Dr. Turegun‟s research and publications cover topics related
to immigration, settlement, integration, and newcomer employment issues.
22 Log files are raw data files recorded by a server to monitor page requests by users, referrer sites, user information,
date and time on a website. Web analytics are the measurement, collection, analysis and reporting of internet data
from log files for the purposes of understanding and optimizing web usage.
14
2.3. Limitations of the methodology
The methodology employed in this study was subject to several limitations identified below.
Cost comparisons between the GTC-IP Initiative and OGD initiatives were not possible due
to significant dissimilarities in the scope of the websites, technical considerations, and web
data capture. This limited the extent to which the efficiency and cost-effectiveness of the
GTC-IP (Evaluation Question 15) could be evaluated in comparison to other initiatives.
A lower than expected number of completions (expected: 50023, obtained: 299) for the online
user survey prevented some sub-group analyses such as using age group, country of birth, and
length of time in Canada. The sub-group analyses would have resulted in a significant margin
of error. Attempts were made to improve the number of completions – for example,
continuous survey monitoring occurred and the length of time the survey was in the field was
extended. Lower response rates among specific groups of intermediaries (e.g., 97 foreign
missions in Canada were invited to complete the survey and only 11 responded) precluded
analyses (due to a resulting high margin of error) by intermediary group and diminished the
ability to make conclusions based on those data alone.
It was not possible to know whether online survey respondents and focus group users of the
GTC-IP were typical of all users of the Portal. Survey respondents and focus group users
generally had positive opinions of the GTC-IP and may reflect a positively-skewed selection
bias. Typically, the analysis of non-response bias consists of a comparison of variables in the
survey sample with those in the sampling frame. This allows for an assessment of the
difference between the true population (i.e., all users or non-users) and the survey sample
(i.e., users who completed the survey, and those that attended focus groups). As no
information was available regarding the survey population (users of the GTC-IP and non-
users of the GTC-IP), it was impossible to assess whether the survey and focus group
participants form a representative sample of all GTC-IP users and non-users. However, user
survey respondent characteristics align with CIC‟s past experience in immigration and
respondent patterns. As well, triangulation of several lines of evidence was undertaken in
order to improve confidence in the survey findings.
Some focus group participants were unable to clearly recall experiences coming to Canada
and using Internet information sources, which may have resulted in some not providing as
much relevant information as others, and some of the barriers and challenges faced may not
have been recalled. Fortunately, the impact of this limitation was minimized with the
inclusion of over 90 focus group participants who had experience with the GTC-IP or
immigrated to Canada within five years prior to focus group participation and the majority of
these had clear recall.
Inconsistencies were noted in web reporting and data collection techniques across the
Initiative‟s partners (WiC, GTC and provinces and territories). This resulted in limited
comparable data available for analysis purposes between GTC and WiC. Comparisons were
not sought between P/T partners; however, it did pose a challenge in the evaluation to have
to report on a variety of metrics over various timeframes and collected at different intervals.
Attempting to mitigate this, additional analytics and log files were requested but these were
23The expected number of online user survey completes was 500 as this number had been achieved in past surveys
conducted on the GTC site over the same time period.
15
not available from all partners. The use of different analytics packages (which used different
algorithms) to determine number of visits, unique visitors, and other key metrics among
Portal partners also resulted in difficulty analyzing the effectiveness of promotional activities
based on website usage. As well, only a few provinces and territories were able to provide
longitudinal data which resulted in an inability to perform time trend analyses to determine
the effectiveness of the provincial and territorial portals over time.
16
3. Results and key findings
This section presents the key findings of the Evaluation of the Going to Canada Immigration
Portal.
The GTC-IP addresses a continued and demonstrated need among its target audience.
Prior to the GTC-IP Initiative, information and services aimed at helping immigrants make
informed decisions about coming to Canada and settling was fragmented and, to a large extent,
alternative sources of online information still exist outside of the Portal Initiative. These sources
are provided by both the private and public sectors, such as through immigration consultants,
online reference sites, private individual sites, Service Canada, SPOs, and the main CIC website.
The creation of the GTC-IP Initiative was intended to address the perceived problem of a
dispersed landscape of information providers by unifying much of the information under the
umbrella of a single collaborative effort. It was felt that this would help streamline much of the
information being served to the target audience.
The need for streamlined, bundled and easily accessible information on immigration, settlement,
education and employment, continues today. For example, a 2008 usability study of the GTC
website found that site users wanted information on a variety of topics, including eligibility
requirements, available jobs and detailed and up-to-date labour market information - all of which
are offered on the GTC-IP.24 This need was seen in another survey of recent immigrants
commissioned by British Columbia, in which it was revealed that the most sought after
information by immigrants and newcomers included a wide collection of knowledge related to
Canada‟s “health care, education, employment (labour market information), English language
training, [and] job search…25 . The study in BC also offered evidence to suggest that there is a
need for, “authoritative information for front-line staff [to help] support clients through the
immigration and integration process.26”
Furthermore, in his 2009 report to federal, provincial and territorial ministers, the chair of the
Advisory Panel on Labour Market Information, Don Drummond, outlined how labour market
information (LMI) was essential to Canada's economic growth and efficiency, allowing the
country to better respond to the growing information and planning needs of employers, workers
and learners.27 Drummond‟s report spoke of the benefits and necessity of a single, free, and fully
accessible LMI platform in Canada – not only for potential immigrants and newcomers, but also
for Canadian citizens. In this regard, the report recognized HRSDC‟s WiC tool as having a
central place in the delivery of that information. The report went on to categorize WiC as
Canada‟s chief source and most comprehensive platform for LMI.
17
This was supported by most key informants who felt there is a continued need for the Portal
because it is complete, comprehensive, and more efficient and effective than the alternative
sources of information noted above. The GTC-IP was generally described in interviews as a
source which contains a greater breadth and scope of information than most alternatives. In
effect, key informants referred to the GTC-IP as a primary source of information that reduces
the need to visit multiple sites or consult multiple sources of information. As well, key
informants felt that other sources of information may not provide the same degree of accuracy or
reliability as is provided on the GTC-IP.
Focus group participants who had used the GTC-IP echoed the feelings of key informants
regarding the continued need for a Portal by citing specific examples. Users expressed how the
GTC-IP provided them with desired information on Canadian climate, culture, and what to
expect when arriving in Canada. While some focus group participants reported finding this
information on the CIC main site, they felt that it was not readily available due to the site‟s overly
complex layout and design, and the sophistication of the language it used. Focus group
participants also noted the benefits of having a wide array of reliable information in one place,
rather than spread across multiple, unrelated locations. Upon being shown the GTC-IP during
group sessions, non-users within focus groups generally felt that the information contained on
the Portal would have been useful to them had they known about it – both before and after they
arrived in Canada.
In March 2010, CIC also conducted a Review of Recent Literature on Horizontal Management,
which was prepared in support of the Summative Evaluation of Canada‟s Action Plan Against
Racism (CAPAR). The primary focus of this review was to examine the objectives, rationale, and
strengths of federal government initiatives which involve two or more departments/agencies, in
addition to drawing upon a wider array of literature on collaborative and networking
arrangements. A key message from the review was that horizontal initiatives help governments to
address increasingly complex issues, where no clear solution exists or is manageable by a single
department or stakeholder group.28 As a horizontal initiative, the GTC-IP helps to unify the
efforts of an ever expanding network of stakeholders involved in migration and labour market
issues, in an effort to provide a comprehensive, authoritative online resource for its target
population.
The GTC IP is aligned with CIC and HRSDC’s strategic outcomes and Government of Canada
priorities
The GTC-IP‟s 2007 Project Charter29 outlined five key objectives of the Portal Initiative: to
create an authoritative, comprehensive Internet source for immigrants and newcomers; to
increase awareness of the opportunities, challenges and barriers to immigrate to Canada; to
inform immigrants of the steps necessary to adapt, live and work in Canada; to help immigrants
make local connections in their P/T or community; and to build effective long-term relationships
with stakeholders and partners of the Initiative.
18
These objectives link to the strategic outcomes of both CIC (Strategic Outcome #3 – “Successful
integration of newcomers into society and the promotion of Canadian Citizenship”) and HRSDC (Strategic
Outcome – “A skilled, adaptable and inclusive labour force and an efficient labour market”). These strategic
outcomes are further articulated in CIC and HRSDC‟s respective 2010-11 Reports on Plans and
Priorities (RPPs). In support of this finding, all HRSDC and CIC key informants stated that the
Portal helps new immigrants adapt to and participate more fully in the workforce, helps to
support settlement objectives and/or facilitates the integration of newcomers.
In addition to being highly aligned with CIC and HRSDC strategic outcomes, the GTC-IP was
found to be aligned with priorities of the Government of Canada. For example, the GTC-IP is
linked to the Government of Canada‟s Advantage Canada commitments, which were reiterated in
the October 2007 Speech from the Throne.30 The GTC-IP also supports commitments made in
the Government of Canada‟s Budget 2007 – namely, to attract and retain skilled immigrants by
better informing potential newcomers of the realities of living and working in Canada. The GTC-
IP supports Canada‟s Economic Action Plan (January 2009) which includes a commitment to
help newcomers get their credentials recognized faster so that they can more swiftly pursue
appropriate pathways to employment. Simultaneously, the WiC site works to strengthen the
implementation of the FQR Framework by developing and integrating specialized information
on qualification recognition processes for different occupations and in different jurisdictions.
This finding was supported by all CIC and HRSDC key informants who stated that the GTC-IP
supports GoC priorities by providing a link between new Canadians and Canada‟s economic
priorities, including Canada‟s economic action plan, and by addressing the need for a larger
skilled workforce. Interviewees also reported that the GTC-IP Initiative supports Canada‟s
settlement objectives.
Almost all key informants who were asked this question said that the federal government‟s
responsibility over the GTC-IP is appropriate. The shared federal and provincial/territorial role
with respect to immigration, which is enshrined in the Canadian Constitution, and the need for a
single information source that provides national-level content and data, were frequently cited as
reasons to support the federal role in the GTC-IP. It was felt that competition between
provinces and territories for immigrants necessitates the provision of neutral, pan-Canadian
information and coordination of national information. It was noted that provinces and territories
are well-placed to provide more specific, P/T, and local information for newcomers.
Most key informants also indicated that delivery of GTC-IP content by alternative sources would
not be as effective, as they feel that they do not currently provide the same level of information
immigrants and newcomers seek. Focus group participants also spoke of the trustworthiness of
information delivered by an authoritative source such as the Government of Canada.
The duplication and overlap of content between the CIC main site and the GTC-IP challenges
the relevance of the Going to Canada site.
19
In order to leverage the significant user population of the CIC main site, CIC has sought to
replicate much of the content from the GTC site on the CIC main site. Comparing the
information on the CIC main site and the GTC-IP sites, this evaluation found 5 major areas of
overlap:
1. Information related to the theme of “Welcome to Canada,” which includes sections on
preparing potential newcomers to get to know Canadian customs, geography, climate,
laws, etc; moving to Canada; and what one can expect during their first days here;
2. Information related to immigration issues such as entry requirements, adoption, and
travelling to Canada;
3. Information on visiting Canada, specifically dealing with visa requirements;
4. Information on studying and working in Canada as a foreign student;
5. Information on finding a job in Canada, which links to the WiC site and WiC tool.
A perception of duplication between GTC and the CIC main site has also been noted by Portal
partners. For instance, many key informant interviewees felt that at least some of the content
available on the Portal is duplicated elsewhere, with almost all of those who felt there was
duplication citing the CIC main site as the chief source. Overall, a few of the interviewees were
also inclined to believe that most, if not all, of the settlement information available on GTC can
be found on the CIC main site, with one respondent claiming: “the Portal (meaning GTC) is
living in the CIC site‟s shadow”. Interviewees were concerned that because the majority of the
GTC‟s content is provided on a website with much higher profile and traffic, users would be less
likely to seek out and use the GTC, thus limiting its usefulness. Users in focus groups also
mentioned being confused between the CIC main site and GTC as they both offer similar
content. As they were more familiar with the CIC site, focus group participants chose it over the
GTC when seeking information.
There is also evidence to suggest that the GTC site‟s target audience has visited and obtained the
information they require within the above five areas of duplication by going to the CIC main site
prior to, in conjunction with, or in place of the Portal. The CIC main site was the primary source
of information for 84% (104 of 125) of intermediary survey respondents when providing services
to potential immigrants and/or newcomers to Canada. As well, in the user survey, respondents
were asked to identify other sources they had used to obtain information on immigrating,
working, settling, visiting, and studying in Canada, as well as on Canada‟s provinces and
territories. Many indicated that they had used other GoC websites to access information
contained on the GTC, with the CIC site being mentioned often. To give one example, 51% (47
of 92) of surveyed users said that they accessed information on immigrating to Canada on
another GoC website, with a further 89% of those (42 of 47) indicating the CIC main site as the
source.
During focus group sessions with users, respondents underscored how it was easier to find and
locate the CIC main site when using a search engine to find information on immigration and
settlement issues. Non-users in focus groups who had heard of the Portal also reported that the
CIC main website was recommended to them as an information source – not the GTC-IP. Many
non-users had also never even heard of the GTC-IP to begin with and recommended that there
should be a more prominent link from the CIC main site to the GTC site in order to highlight
GTC‟s content.
20
Compounding the challenges of duplicated information between GTC and the CIC main site is
the fact that the number of visitors to the CIC departmental site is significantly greater than the
number of visits to the GTC-IP. Log file analysis showed that in 2009, on average, the monthly
unique visitor count for the main CIC site was approximately 3.5 million, compared to an average
of approximately 248,000 unique visitors for the GTC-IP (WiC and GTC sites combined from
January-April, 2010), as noted in section 2.2.3. The impact of these figures on their own should
be tempered by considerations of the differences between site users, and the content and history
of these two sources. The CIC site has been running longer, has had more exposure, and holds
information that is used by a broader client group than those who visit the Portal.
Nevertheless, the duplication and overlap in content, along with evidence of the CIC main site‟s
dominant web presence (as perceived by users and non-users of the GTC-IP) and the significant
differences in web traffic between the Portal and CIC main site, points to ongoing challenges to
the primacy of information contained on the Portal moving forward. Compared to the GTC site,
however, the WiC site has not been affected by issues of duplication. Content related to “finding
work” on the CIC site is clearly and directly linked back to the WiC site. The WiC tool is also
designed to be versatile and to be shared across different sites – including the CIC main site.
Accordingly, the CIC site only accounts for 6% of referrals to GTC, whereas it accounts for 32%
of the referrals to WiC as shown in log file analysis.
3.2. Performance
3.2.1. Governance structure
21
Governance between CIC and HRSDC
The governance between CIC and HRSDC has improved over time but issues with governance
remain.
The original horizontal governance structure for the GTC-IP Initiative was articulated in the
2007 Project Charter as well as the Results-Based Management and Accountability Framework.
This model included a variety of committees and working groups responsible for providing
coordination and oversight with respect to the GTC-IP. None of these committees and working
groups had a Terms of Reference to guide their operation. Almost half of the interviewees from
CIC and one from HRSDC reported issues with the former governance structure, stating it was
overly complex for such a small initiative and was difficult to implement. In 2010, CIC and
HRSDC redesigned their committee oversight structure to be more streamlined. Interviewees
report that this new model has improved the effectiveness of the GTC-IP‟s governance.
This notwithstanding, the current governance between CIC and HRSDC was considered
effective by only a few CIC and HRSDC key informants. However, the majority of remaining
interviewees did not provide a clear response regarding the effectiveness of the current
governance structure. As mentioned, many focused on the former structure and others were
neutral in their response. Only one individual reported that the current structure was not
effective, attributing this to a lack of common vision between CIC and HRSDC.
Other governance-related concerns raised by interviewees included a lack of clarity surrounding
future directions of the Initiative and in the roles and responsibilities of both departments.
However, some interviewees believed that an upcoming Letter of Understanding (LOU) to be
signed by CIC and HRSDC will serve as a solution to these issues. For instance, the LOU is
expected to clarify roles specific to each department in terms of content and partnership
development. The LOU also provides information on the new horizontal committee structure
for the Initiative.
The LOU also states that CIC will continue to host FPT portal workshops and coordinate FPT
conferences. HRSDC will host a yearly labour market information workshop for Working in
Canada Tool partners. Each department will be a standing participant at each other‟s event to
ensure proper policy linkages are developed between settlement and labour market related
information.
The above noted measures are supported by the Horizontal Literature Review which outlines the
criteria for effective horizontal management.
Weaknesses in CIC’s internal governance structure were identified in the evaluation but recent
improvements were also noted
Historically, there has been a lack of clarity around CIC‟s governance structure. There were no
documents provided for the evaluation that outlined CIC‟s originally intended internal
governance structure for the GTC-IP. Key informant interviews also did not provide clarity on
the former structure, with some interviewees stating that Integration Branch led the Initiative at
CIC, while others referred to joint leadership between Integration and OMC Branches. This
confusion could have been due, in part, to the March, 2006 departmental reorganization that led
to the creation of the OMC Branch. The reorganization was implemented in order to “create
22
coherent and consistent program delivery throughout Canada and abroad”. Prior to that
reorganization, Integration Branch‟s policy and operations units were housed together in one
team. This split between policy and operations may have contributed to a lack of clarity around
CIC‟s internal governance structure, which has continued to the present day.
Most of the CIC respondents reported that the internal governance of the Portal has been
fragmented, with responsibility for components of the GTC-IP resting in three different
branches within CIC. Just over half of the CIC interviewees cited a combination of blurred lines
of communication, undefined roles and responsibilities, differing opinions of three Branches
leading to difficulties in decision-making, a lack of leadership and an uneven distribution of
resources as both the cause and effect of CIC governance structure problems to date. A few CIC
respondents either stated that governance has not been an issue or did not provide a response.
Several HRSDC key informants also commented that CIC‟s governance structure lacked clarity as
a result of undefined roles and responsibilities.
Issues with CIC‟s internal governance of the GTC-IP were articulated in a deck titled Management
of the Going to Canada Immigration Portal project within CIC, presented by Integration Branch at a
Portal Directors General meeting in June, 2008. That document stated that the structure needed
to improve because there was an overall lack of accountability due to decentralization of
authority, a significant overlap of responsibilities, and deficient coordination of the Initiative
overall, with no one taking ownership. The document recommended that project management
for the GTC-IP within CIC be centralized and led by a branch with expertise in web and
communications. It also recommended that roles and responsibilities be defined by expertise and
function.
These recommendations were partially implemented recently, with e-Communications Branch
taking over as CIC lead of the GTC-IP. As well, an internal CIC Letter of Understanding has
been drafted (for signature by e-Communications, OMC, Integration, Finance, and IMTB
Branches), which more clearly outlines the roles and responsibilities of each CIC Branch
implicated in the GTC-IP. Regular CIC meetings at various levels within the department were
also outlined in this LOU, although Terms of Reference for these committees have yet to be
developed. Almost half of the key informants from CIC expressed optimism about the
functioning of CIC‟s governance stemming from the LOU, and have cited recent improvements.
However, this LOU has yet to be signed by the affected branches.
HRSDC’s internal governance structure has been effective and has allowed for enhanced
creativity and innovation in the development of WiC content and tools.
The governance model currently employed by HRSDC involves one team, working in one
branch. It was viewed positively by both CIC and HRSDC respondents, with no interviewees
reporting that the HRSDC model was ineffective. Almost half of the CIC key informants
reported that HRSDC‟s governance model is more effective than the one used by CIC. The
approach whereby most WiC-related activities are performed by a single group of staff dedicated
to Working in Canada, has fostered an atmosphere which encourages creativity, innovation and
efficiencies according to key informants. For example, all of the systems work on the WiC
website is conducted “in-house,” which facilitates implementation of updates, as well as planning
and coordination with other branches within HRSDC and with other GTC-IP partners.
23
The effectiveness of this type of structure was corroborated by other government departments32
(OGDs) and agencies who operate and maintain websites targeted at potential immigrants and
newcomers and who were interviewed as part of the evaluation. These OGDs were asked to
comment on their own internal governance structure in order to better understand the positive
and negative features of different governance models. Although responses regarding the
governance structure of their portals varied, those representatives generally commented that
working in a single team or unit when engaged in web-based initiatives provides efficiencies from
a resource and cost-based perspective, thus, supporting the governance structure employed by
HRSDC and reiterating one of the challenges encountered by CIC in developing and maintaining
GTC.
Roles and responsibilities of the PTs are generally clear; however additional clarity is
required around reporting
The Contribution Agreements (CAs) in place with the partnering provinces and territories
articulate the roles and responsibilities of CIC and the P/Ts related to the GTC-IP Initiative.
Specified in these agreements are details around funding, reporting requirements, and
collaboration. The majority of provincial and territorial representatives reported that their roles
and responsibilities are clear and that this is supported through ongoing collaboration and
information sharing by Portal partners.
However, in spite of a clear articulation of roles and responsibilities within the CAs, a number of
interviewees from CIC and HRSDC and one provincial/territorial representative reported a lack
of clarity in roles and responsibilities of the P/Ts stemming from differences across contribution
agreements and the absence of a common goal and vision among P/Ts. One difference noted
was that some CAs clearly identify staffing as an allowable project expense while others do not.
Given the nature of F/P/T relations, OMC Branch negotiates the CAs with individual P/Ts,
which can result in differences across these documents. In the case of salary dollars identified
above, in fact, salary and/or contract expenses related to direct and indirect projects were
allowable expenses for all partners although this was not articulated in the individual CAs.
While differences across contribution agreements likely led to some of the confusion regarding
the roles and responsibilities of the P/Ts, it is likely that reporting on expenditures and outputs
was a contributing factor as well. Each contribution agreement stipulates that anticipated
spending by the provinces and territories should be reported annually, detailing activities,
deliverables, and costs related to specific projects. Comparing the Expenditure Reports across the
P/Ts, this evaluation found that despite some P/Ts‟ contribution agreements being identical on
this aspect, reporting on eligible costs differed in level of detail, with some P/Ts describing the
individual costs of specific items and deliverables, while others gave lump sums with an attached
list of activities, connected to larger activities. A few respondents from CIC also noted a lack of
clarity regarding P/T projects and outputs achieved as a result of GTC-IP funding.
As well, a September, 2009 deck titled Contribution Agreements Reporting and Monitoring, which was
presented at the St. John‟s bi-annual workshop, provides evidence of another difference. The
32OGD websites included Canada Mortgage and Housing Corporation (CMHC); Foreign Credentials Referral
Office, Citizenship and Immigration Canada; Office of Consumer Affairs, Industry Canada; Aboriginal Canada
Portal (Indian and Northern Affairs Canada); Portal of the Government of Quebec.
24
CAs allow for a variety of invoicing frequencies across P/Ts, with some choosing to produce
expenditure reports annually while others report semi-annually or quarterly. This deck also
highlighted issues with timeliness of submission of required reporting by P/Ts; however, the
document review conducted as part of this evaluation showed that this has been improving
recently.
Communications structures of the GTC IP, such as the bi annual workshop, regular meetings,
and the WiC Wiki, are productive and effective.
Interviewees spoke of a variety of communication mechanisms, both formal and informal, that
are in place within the GTC-IP Initiative. Most interviewees highlighted the bi-annual workshops,
while several others mentioned the WiC Wiki, meetings (both in-person and via teleconference),
and e-mails between partners. Each of these mechanisms was seen as effective by most
respondents, with the bi-annual workshops, followed by the Wiki, being seen as the most
effective. Records provided by CIC and HRSDC (e.g., meeting agendas, minutes, and
documented decisions at Portal Director‟s meetings) and a review of the Wiki further
demonstrate the effectiveness of communications between Portal partners.
Partners of the GTC-IP were asked how communication between partners could be improved.
Despite an overall satisfaction with the current mechanisms, there were a few suggestions made,
which included increased use of organizing materials such as agendas and calendars of
events/projects, the use of new collaborative tools such as virtual meetings/videoconference, and
an increase in opportunities for multilateral communication through mini-workshops at the bi-
annual workshop or regular conference calls with PTs.
A primary strength in the delivery of the GTC IP Initiative lies in the effectiveness of
collaboration among partners, with the bi annual workshops cited as a best practice in
information sharing.
25
The most highly regarded collaborative effort has been the bi-annual workshops, to which all
Portal partners are invited. The workshops were acknowledged as the key to the development of
partnerships and relationships; sharing of information resources and tools among partners; and
the development of common goals and strategies. These key informants spoke highly of the way
in which the workshops generated a positive atmosphere around Portal projects, have “encouraged
innovation and creativity” among Portal partners and led to the free exchange of specific tools and
information, including the WiC tool, interactive maps, software codes and programming, and
performance measures and practices. This free exchange of tools and knowledge among GTC-IP
partners is a key benefit to ongoing collaboration by partners and stakeholders in the Initiative.
Table 3-1 provides an overview of tools and information that have been shared among GTC-IP
partners. Sharing of these tools increases the level of sophistication of all partner Portals and
creates considerable cost savings for partners, as the development of these tools on an individual
site basis would be cost-prohibitive.
Table 3-1: Examples of tools and information that have been shared among GTC-IP
partners
26
The majority of PTs feel that they have some role in setting priorities for the GTC-IP.
Most P/T interviewees said that they have a voice in setting priorities for the GTC-IP, and are
involved in decisions that will impact their province or territory. A number of examples of where
input was solicited were cited by these respondents, including a two-way dialogue when
establishing the contribution agreements between CIC and the P/Ts, having the opportunity to
provide comments and input on the GTC and WiC websites, and being provided a venue to raise
questions and concerns (e.g., meetings, emails). Of note is that two P/T respondents felt they
had more of a role in decision-making with HRSDC than with CIC.
Although generally accurate and complete, CIC’s budget allocations and expenditures,
including those by provinces and territories, have not been tracked in a consistent way. There
were no issues identified with HRSDC’s tracking of expenditures.
Within CIC
Table 3-2 shows financial information on CIC‟s component of the Portal. Information broken
down by individual branch within CIC was not available for the purposes of the evaluation. The
allocations presented in the table are total allocations to CIC and are consistent with the amounts
reflected in the Treasury Board submission for the GTC-IP. The funding reflects initially
approved A-base funding and does not take into account various government-wide operating
budget reductions and administrative measures that the department had to absorb over the last
few years.
The expenditures included in the table are those that have been tracked using SAP, the
department‟s financial coding system. However, expenditures reflect what has been coded under
the Internal Orders (IO) in the financial system only. Therefore, if the IO is not used in the
coding, expenditures will not be tracked. Portal funding was allocated to various sectors and
branches in the department but was not tracked consistently throughout the department.
Therefore, we are unable to say with any certainty how much of the total allocation was spent on
the Portal Initiative.
Table 3-2 also shows the percentages of total Portal untracked funding by CIC as 79% in 2007-
08, 68% in 2008-09, and 62% in 2009-10. Internal records from CIC‟s branches implicated in the
27
Portal Initiative show that a variety of projects were completed on the Portal and provide
evidence that the expenditures are much higher than indicated by the IO coding. For example, in
2007-08, the department engaged in web testing and public opinion research (POR) on the
Going to Canada Site as well as development of the Entry Requirements Tool. In 2008-09, the
Entry Requirements Tool was finalized and the Service Provider Organization (SPO) tool was
developed. Plain language benchmarking and additional POR was also conducted on the Portal in
that year. In 2009-10, Google ad words campaigns and a visitor path analysis were conducted on
the Portal. These accomplishments are not all reflected in the current Portal expenditures as they
were likely not tracked using the Portal IO.
As noted above, the Portal was transferred to CIC from DFAIT in 2007 and in the first year of
this transition period, the department‟s capacity to deliver the program may not have been fully
realized. As well, full resources may not have been expended on the Portal Initiative over the last
three years due to the anticipated integration of the Portal into CIC‟s main site.
Additional information was provided by CIC‟s branches on money that was not tracked as part
of the Portal Initiative, although the exact amount of funds that were actually lapsed cannot be
provided. Several projects were planned and money put aside for their implementation. However,
the organizations responsible for completing these projects were not able to do so within the
fiscal year and therefore money may have lapsed due to project delays or cancellations. Examples
of these projects are the Interactive Map Project, the Cyber Mentorship Project and Site Search.
28
Table 3-2: Funds allocated and spent by CIC (2007-08 to 2009-10)
2007-08 2008-09 2009-10 Total
Partner Tracked Tracked Tracked Allocated
Allocated Variance Allocated Variance Allocated Variance (2007-2010)
Expenditure Expenditure Expenditure
CIC TOTAL $2,153,184 $448,706 $1,704,478 $1,703,184 $547,249 $1,155,935 $1,803,289 $690,762 $1,112,527 $5,659,657
Note: Amounts provided in the total expenditure columns are the amounts captured in SAP and do not reflect actual expenditures.
Within HRSDC
Table 3-3 outlines HRSDC‟s Portal allocations and expenditures for fiscal years 2007-2010. The numbers provided by HRSDC Finance were
received for the evaluation in a timely manner, without indication of any tracking issues. According to the records, HRSDC spent an average of
93% of their allocated funding over three fiscal years.
29
When P/Ts lapse money, these funds were automatically returned to CIC for use in other programs or areas of activity, unless otherwise
negotiated with each province or territory to be re-profiled to the following fiscal year. The most commonly cited reasons by the provinces and
territories for those lapses in funding were delays in portal development, or the delay of specific projects or activities (e.g., delays in developing or
launching interactive maps). This could be expected as not all of the provincial and territorial portals were fully functional over the period
covered by the evaluation. Issues in one area of portal development could have hindered implementation of other planned activities, depending
on the stage of portal implementation. Issues related to lapsed and re-profiled P/T portal money is reflected in Table 3-4. According to these
records, in 2007-08 the provinces and territories spent only about 50% of their allocated funding. In the subsequent years, the P/Ts collectively
overspent by 13% of allocations in 2008-09 and 6% in 2009-10.
Table 3-4: Funds allocated and spent by provinces and territories (2007-08 to 2009-10)
2007-08 2008-09 2009-10 Total
Partner Tracked Tracked Tracked Allocated
Allocated Variance Allocated Variance Allocated Variance (2007-2010)
Expenditure Expenditure Expenditure
British Columbia $660,682 $219,687 $440,995 $934,637 $990,438 ($55,801) $903,027 $735,525 $167,502 $2,498,346
Alberta $399,982 $347,560 $52,422 $555,522 $555,522 $0 $561,712 $561,712 $0 $1,517,216
Saskatchewan $220,457 $28,374 $192,083 $247,491 $170,555 $76,936 $254,400 $254,400 $0 $722,348
Manitoba $279,002 $190,000 $89,002 $363,617 $524,000 ($160,383) $373,972 $271,000 $102,972 $1,016,591
Ontario $2,000,000 $933,267 $1,066,733 $2,000,000 $3,216,382 ($1,216,382) $2,000,000 $3,108,092 ($1,108,092) $6,000,000
New Brunswick $210,235 $192,827 $17,408 $222,766 $204,129 $18,637 $226,225 $225,131 $1,094 $659,226
Nova Scotia $219,676 $91,255 $128,421 $238,844 $72,308 $166,536 $240,748 $195,396 $45,352 $699,268
Prince Edward Island $203,052 $170,424 $32,628 $207,853 $207,784 $69 $211,111 $199,755 $11,356 $622,016
Newfoundland &
$205,253 $205,000 $253 $211,332 $211,332 $0 $210,855 $199,747 $116,855 $627,440
Labrador
Yukon $200,667 $123,288 $77,379 $201,165 $141,419 $59,746 $201,217 $139,707 $61,510
Nunavut $200,085 $0 $200,085 $200,152 $0 $$200,152 $200,208 $0 $200,208 $600,445
Northwest Territories $200,909 $26,079 $174,830 $201,621 $23,780 $177,841 $201,521 $54,858 $146,663 $604,051
Total (provinces &
territories) $5,000,000 $2,527,761 $2,472,239 $5,585,000 $6,317,649 ($732,649) $5,584,996 $5,945,323 ($360,327) $16,169,996
30
Over time, GTC-IP funding allocations within CIC and to provinces and territories have become
more responsive to the needs of the Initiative
Financial records showed that in fiscal years 2007-08 and 2009-10, the four CIC branches
implicated in the GTC-IP Initiative (overall) did not track between 62% and 79% of
expenditures. As well, just over half of the interviewees from CIC commented that resources
allocated to the Portal were sometimes used for other projects. This may be due in part to a
reallocation of a portion of Portal funding to branches within CIC that have not been implicated
in the GTC-IP Initiative (e.g., FCRO, Immigration Branch, SIO) to date.
In the spring of 2010, a review of Portal allocations within CIC was conducted in order to
address these issues. The result of that review was a restructuring of funding allocations to ensure
that funding is distributed more appropriately to three of the four Branches within CIC (e-
Communications, Integration and OMC) and is in accordance with the activities and level of
staffing required by each. Prior to the current fiscal year, IMTB Branch had permanent funding
associated with the Portal. In future years this will not be the case as it was recognized that
IMTB‟s functions are project related and are not consistently required from year to year.
As noted above, provinces and territories are in differing stages of development of their web
portals, which can create varying needs across partners. OMC has ensured that allocations to
P/Ts have been flexible in order to accommodate these differences. In some cases, funding that
will not be used by one province can be moved to another PT that has the need for additional
funding to complete a project or to fund a municipal portal, for example. As well, when project
delays occur, causing funds to lapse, the department can re-allocate funds in order to provide this
money in subsequent fiscal years. In the case of Ontario, OMC made special arrangements to
accommodate delays in the flow of funding as a result of signing the Canada-Ontario
Immigration Agreement (COIA). This flexibility allowed Ontario to spend its entire allocation
over the five years of the program.
These activities create a funding model that is more responsive to the needs of partners and
allows for completion of projects that contribute to overall program outcomes.
Evidence suggests that the GTC-IP is being used as an information source; however, this is
primarily focused on working and immigration related content.
31
250,000
Figure 3-1: Growth in usage of
the WiC, 2008-2010 195,429
(# of unique visitors 200,000
in March of each
year) 150,000 128,983
100,000
As shown in Figure 3-1, the WiC 45,199
website has experienced considerable 50,000
growth over the last three years
(2008 to 2010), growing from 45,199 0
unique visitors in March 2008 to March 2008 March 2009 March 2010
128,983 unique visitors in March Note: This figure demonstrates the growth in number of unique visitors to
2009 and to 195,429 unique visitors the WiC site over a three year period. These numbers were obtained from
WiC site web analytics.
in March 2010.33
As well, web data from Google Urchin show consistent use of the GTC site over time with
191,620 page views recorded in March 2009 and 191,587 in March 2010.
Intermediaries who provide services to new and prospective immigrants are also using the GTC-
IP, as more than two-thirds (69% or 86 respondents) of those surveyed (N=125) said that they
have used either the GTC website or the WiC website, with 54 of these respondents reporting
using both the GTC website and the WiC website. In addition, 16 respondents used only the
GTC site and 16 used only the WiC sites. Moreover, almost half (47%) of the 70 intermediaries
who visited the GTC site indicated they had done so within the last month. Seventy-six percent
of the intermediaries who have visited the GTC and 76% who have visited the WiC website
indicated they have used or referenced information from the site in their work.
As well, results from the online survey of GTC-IP users found that 36% (107 of 299) of
respondents who had visited the GTC-IP did so at least 5 or more times in the last year. In terms
of continued relevance, these figures give an indication of the frequency of use by those who
have recently accessed the GTC-IP. Furthermore, 122 of 139 users (88%) reported that they
would recommend the GTC-IP to others – suggesting users found the sites useful.
32
Web analytics provided by CIC also show that the SPO tool was being accessed by users of the
GTC-IP. For example, in September 2010, approximately 10% of all visits to the SPO tool (1,559
visits) came from the Going to Canada website.
P/T portals are being used, but differences in data collection and reporting techniques have
limited the analyses of usage data among the P/Ts.
A wide range of practices in web data collection among the provinces and territories prohibited a
complete and comprehensive analysis of usage of their portals. For example, various metrics are
collected and reported differently among the provinces and territories (e.g., some
provinces/territories only collect the number of visits, while others collect numbers of unique
visitors, while still others collect page views). Also, differing time periods of data were provided
(e.g., some provinces/territories provided multiple years of data, while others provided only days
or months of data), which restricted analyses.
34These content areas are comprised of page views of a list of the top 25 most viewed pages on GTC in 2009, which
account for 70% of all page views on the GTC site.
33
Nevertheless, web data does tend to suggest that the provincial and territorial portals are being
used. For example, in the first three months of calendar year 2010, the New Brunswick portal
averaged 7,807 visits per month35, while the Saskatchewan portal averaged 42,209 visits per
month in 2010. Available data also show that usage of some provincial and territorial portals is
increasing over time. As shown in Table 3-5, available trend data show that the British Columbia,
Alberta, Manitoba, Ontario and Yukon portals all generated increased use over time (as measured
by the number of unique visitors per month or the number of visits per month).
Table 3-5: Use of the provincial/territorial portals over time (2008 to 2010)*
Province / territory 2008 2009 2010
Unique Visitors per Month
A lack of promotion and awareness among the target audience has limited the use of GTC
while the WiC website has benefitted from more active outreach campaigns.
Although the GTC-IP is being accessed and used as an information source, the profile and
awareness of the GTC website in contrast to the WiC site is low.
Awareness
Focus group participants who had previously used the GTC reported first finding the website by
conducting searches in Google. They would use the popular search engine to find different
combinations of keywords related to immigrating, moving, jobs, etc. However, these keywords
did not push the Portal to the top of the search results as GTC‟s page rank is relatively low in
Google. Using the terms „Canada and Immigration‟ yielded the GTC website as the 98th result,
while „Immigration and Canada‟ yielded the GTC website as the 93rd result. Using these same
words, the CIC website was ranked first in both instances. Given that Google displays only 10-15
links per search page, the GTC typically would not show up until several pages into the search
results. In comparison, the WiC website was listed first when searching the terms „Working and
Canada‟ on Google, likely facilitated by its extensive search engine optimization.
35 Data calculated using web analytics reports data provided by New Brunswick.
36 Data was provided by B.C. in the form of web analytics data for years 2008-2010. The number of monthly average
unique visitors for 2008 was calculated using available web analytic figures from the months Apr-Dec, 2008. For
2009 and 2010, full calendar years were used.
37 Data for Manitoba was gathered from web analytic reports for calendar years 2008-2010.
38 Data Yukon was calculated using analysis of log files for 2009-2010 calendar years
39 Data for Alberta was gathered from web analytic reports for fiscal years 2008-09 and 2009-10.
40 Data for Ontario was gathered from web analytic reports for fiscal years 2007-2008, 2008-09 and 2009-10.
34
However, the terms „Settling in Canada‟ and „Visiting in Canada‟ yielded much better results –
ranking the GTC 3rd for each of these terms (the CIC main site was still ranked first). The
overall impact of these search terms is lessened, however, given that information on settling and
visiting Canada are not the most sought after content on the GTC site. Other problems affected
the ability of users to find GTC. For instance, searching directly for the „Going to Canada
Immigration Portal‟ in Google yielded the GTC site‟s link which read, „Welcome‟ rather than
displaying the name of the website (this issue was recently fixed). Search engine optimization
issues such as this have likely attributed to low usage of the GTC-IP. Of the 31% (39 of 125) of
intermediaries who indicated they had never visited the portal, roughly half (25 respondents) said
that they had not done so because they had never heard about it. Non-users in the focus groups
sessions also indicated they had not heard of the Portal and those who had, could not find a link
to it.
Online users who completed the survey cited the use of a search engine (e.g., Google, Yahoo,
etc.) most often as their means of finding the GTC-IP (154 of 299, or 52%), followed by those
who found the GTC-IP through a referral from a friend (11%) and those who found the site
from a link on a Canadian provincial or territorial website on immigration (11%). In comparison,
intermediaries were more likely to find the Going to Canada site (63%, 44 of 70) and the
Working in Canada site (57%, 40 of 70) through the CIC website.
Promotion
Compared to the WiC website, there has been considerably less promotion of the GTC website,
particularly in recent years. Documents show that CIC has participated in a variety of conferences
and promotional activities related to the GTC website, most recently attending the Metropolis
Conference in April 2008. CIC has also engaged in other promotional activities, including
distribution of bookmarks, pens, and pamphlets/leaflets. It appears that no promotional activities
have occurred since 2008 with the exception of some recent Google ad-word campaigns, which
have been met with varying degrees of success according to CIC key informants. Many key
informants attributed the lack of promotion to the anticipated merger of GTC website content
with the CIC website.
For the WiC website, HRSDC has engaged in a number of promotional campaigns, the most
prominent of which has been the use of social media, including Twitter, Facebook, and
YouTube.
Twitter: Launched in November 2008, the Working in Canada Twitter feeds consists of
three channels, including channels in English, French and Chinese.41
Facebook: Launched in February 2009 (in both English and French), the English Working in
Canada Facebook page had over 1,300 unique fans from January to September 2010, while
the French Working in Canada Facebook page had over 530 fans for the same nine- month
period. Available data show that an average of 392 clients per month (January to September
2010) accesses the WiC website via the Working in Canada Facebook pages (representing
approximately 41% of all visitors to the WiC Facebook pages from January to September
2010).42
41 HRSDC Web Presence Audit, Centre of Excellence for Public Sector Marketing, March 2010.
42 InsightsAnalytics Report: Facebook Stats, September 2010.
35
YouTube: Also launched in February 2009, the Working in Canada YouTube pages are
provided in both English and French and now contain over 200 videos. Clients are using the
YouTube pages to reach the WiC website, as 151 clients in April 2010 and 144 in May 2010
reached the WiC website directly via YouTube. As of May 2010, videos on the Working in
Canada YouTube pages have been viewed over 150,000 times in English and over 39,000
times in French since they were first launched in 2009. Those YouTube videos are also
available as streaming content through the WiC reports and the WiC widget.
In addition to its use of social media, HRSDC has also engaged in a number of traditional
promotional campaigns, including attending conferences, developing bookmarks, pamphlets and
other branded items and, like CIC, HRSDC has engaged in Google ad-word campaigns to
increase traffic to the WiC website (the HRSDC Google ad-words campaign resulted in a 50%
growth in users from India, the Philippines, and China). HRSDC‟s promotional activities were
highly regarded by both HRSDC and CIC key informants. According to one HRSDC key
informant, “over 80% of the traffic to the site [WiC website] comes from outside of Canada,”
which is supported by web analytics (79% of visitors accessing the WiC were from outside of
Canada from January to April 2010) and suggests that the WiC website is reaching its target
audiences.
Like the GTC-IP, the provincial and territorial portals have engaged in promotional and
advertisement campaigns, although there is insufficient available data to demonstrate the
effectiveness of those campaigns at this time.
To increase overall awareness and use of their portals, a number of provinces and territories have
engaged in both traditional and non-traditional promotional and advertisement campaigns for
their portals. For example, New Brunswick has developed multiple campaigns designed to
increase awareness and promote their provincial portal. A recent campaign was the „Come Home
to NB‟ contest which targeted people nationally using multiple forms of media (e.g., postcards,
online advertisements). Other provinces, such as Newfoundland and Labrador have provided
direct links to their provincial/territorial portals from their main provincial/territorial
government website as a way of further connecting with new and prospective immigrants.
Alberta has successfully engaged in targeted e-mail advertising campaigns, while the British
Columbia portal executed a marketing campaign to leverage exposure and client access through
the 2010 Winter Games.
Other promotional campaigns, cited by key informants and/or provided in various documents,
include Google ad-word campaigns (undertaken by various provinces or territories); using or
linking to social media (e.g., Facebook, Twitter, etc.); attendance at conferences and workshops;
meetings and presentations to service provider organizations; pamphlets and leaflets; and the
distribution of promotional items (e.g., Newfoundland and Labrador‟s Puffin Stress Ball).
While there are many marketing and promotional initiatives being undertaken by P/Ts, there was
insufficient information available for the evaluation on the results of these initiatives. Therefore,
the effectiveness of these campaigns cannot be determined at this time43.
43The Alberta H-1B campaign to promote Alberta to those in the United States working on H-1B work visas
through webinars and live presentations demonstrated success, as the Alberta Portal received approximately 3 times
the average number of visits during the marketing period. Visits returned to pre-advertisement levels after marketing
ended.
36
3.2.6. Usability and functionality
An important measure in a website (or web portal) evaluation is the extent to which it is usable
and functional for its intended target audience. In this evaluation, focus group participants, online
users, intermediaries, and the subject matter expert generally agreed that the GTC-IP was
functional, user-friendly and easy to navigate.
Online users indicated that it was easy to find the information they were looking for in a number
of areas, including immigrating (83%, 76 of 92), settling (80%, 16 of 20), working (78%, 103 of
132), visiting (77%, 10 of 13), and studying (56%, 9 of 16).
Figure 3-2: Percent of online users who said it was easy to find information
100%
83% 80% 78% 77%
80%
56%
60%
40%
20%
0%
Immigrating Settling Working Visiting Studying
(N=92) (N=20) (N=132) (N=13) (N=16)
In addition, intermediaries who indicated that they had visited the WiC site (n=70) and the GTC
site (n=70) were asked to rank the sites‟ usability and functionality. These attributes were rated on
a scale of 1 to 4 where 1 represented “Does not agree at all” and 4 was “Completely Agree.”
Both of these statements were rated quite highly by intermediaries who used the GTC and the
WiC websites with a mean rating of 3.42 out of 4 for being „logical and user-friendly‟ and 3.22
and 3.27 out of 4 (for the GTC and WiC website respectively) for „information is easy to find‟.
3.27
3.22
37
Focus group participants were asked if they had any suggestions to increase the overall
functionality and navigation of the GTC-IP. Responses included: more drop-down menus on the
Portal; more pictures and images on the site; and providing clearer definitions, descriptions and
examples of job titles that can be searched with the WiC tool (many participants cited difficulties
finding their occupation according to their traditional or historical job title using the WiC tool).
Participants in several focus groups also mentioned that one of the biggest challenges of the
GTC-IP was that when they followed one of the many external links on the Portal (e.g., Canada
Border Services Agency website), it was often difficult to return to the GTC-IP. Despite
suggestions to improve the GTC-IP from online users, intermediaries, and focus group
participants (including users and non-users), most felt the GTC-IP was user-friendly and easy to
navigate.
Web analytics produced for this evaluation demonstrate that very few users of the GTC-IP
experienced errors while using the Portal. For example, between January and April 2010, 92% of
all visits to the GTC website and 97% of visits to the WiC website were error-free (resulting in
the user accessing the requested page uninterrupted). Further analyses of GTC-IP log files
demonstrate that the GTC-IP (including the GTC and WiC websites) has one broken link on
every fourth page that is available to users. This is considered acceptable, as “the average website
has one page in every four containing a broken link.”44 As part of the review of the GTC-IP, the
subject matter expert commented positively on the accessibility (functionality) of the Portal,
including the relatively small number of broken links and the currency of links to external
websites. A study conducted in 2008 on the GTC also found that:
The clarity of the language (85%) and the text size and style (81%) meet with greatest approval. More
than three quarters say they are satisfied with the ease with which the Web site could be navigated.45
Part of the overall functionality and usability of the GTC-IP can be attributed to the rectification
of past issues and errors that have been identified on the GTC website and the WiC website.
Based on available data, at least 121 updates, of varying degrees of significance from minor to
major, have been made to the GTC website since 2008. Previous usability studies have also
demonstrated that barriers to usability have been addressed, such as providing more clarity to the
specific sub-menus of the “Welcome to Canada” menu on the GTC site.46 Further anecdotal
evidence emerged during the course of the evaluation that demonstrates that updates are being
made to the GTC website, including the appearance of the title of the GTC website when
conducting a search in Google, which was changed from a generic and unidentifiable “Welcome!”
to the distinguishable “Going to Canada Immigration Portal – Welcome!” in September 2010.
The provincial and territorial portals are generally functional, user-friendly, and easy to
navigate.
Although very few respondents answered questions related to the provincial and territorial
portals (N=7), almost all who did reported that information on the P/Ts was easy to find. These
findings were reaffirmed by the subject matter expert who found the provincial and territorial
websites to be, in general, accessible and easy to navigate. Additionally, documents provided by
the provinces and territories (BC, AB, SK, ON, NFL, NS, and YK), including usability testing,
38
customer segmentation, and search engine optimization studies, demonstrate that the provincial
and territorial portals are generally usable, functional and easy to navigate, and have undergone
improvements to ensure their target audiences are receiving the information they require in an
easy to use format. Examples of these studies are provided in Table 3-6.
Table 3-6: Studies Commissioned by the Provinces and Territories on their Portals
Province/territory Examples of studies: Commissioned by the provinces and territories
Saskatchewan A 2009 usability study of the Saskatchewan Portal found the following:
The visual layout and design are extremely appealing, and facilitate findability.
The main navigation headers support the categories of information needed most by
potential users, and it is in roughly the order that they would naturally look for the
information. Therefore, the site itself emulates users real world needs.
Users search by content and use the site as it is intended.
Users are comfortable with the links that take them to other sites and do not have
challenges finding their way back.
The overall tone and concept are positive, motivating, and extremely well received by
participants.
AEEL Immigration Portal Testing, Vision Critical Group, November 2009.
39
3.2.7. Quality of information
The GTC-IP provides high quality information for its target audience.
For the purposes of this evaluation, the quality of the GTC-IP was assessed according to three
criteria, including:
1. Relevance: whether information provided was pertinent and significant to its target
audience.
2. Clarity: the manner in which information is presented, such as whether it is readable and
understandable to the target audiences.
3. Currency: the extent to which the information on the GTC-IP is up-to-date.
A 2007 study of GTC (that included eight focus groups and 46 one-on-one interviews in eight
cities) found “widespread agreement that the content is clear and easy to understand.” 47
These findings were supported by the 2010 assessment of the GTC-IP by the subject matter
expert who looked specifically at the relevance, clarity and currency of information presented on
the GTC-IP. Based on his expertise in immigration and settlement content, the subject matter
expert found that the GTC-IP is relevant to its target audience, provides clear information that is
sufficiently up-to-date, easy to read, understandable and at an appropriate language-level for new
and prospective immigrants. No concerns were raised by the subject matter expert about the
content presented on the GTC-IP (the GTC website and the WiC website).
As part of their survey, intermediaries were also asked to comment on a number of specific
attributes related to the quality of the GTC-IP. As shown in Figure 3-4, all three statements
related to quality received high agreement ratings from intermediaries who used the GTC (n=70)
and WiC (n=70) sites, with only marginal differences observed between the GTC and WiC
websites. Specifically, intermediaries agreed that information on the GTC-IP is easy to
understand (3.13 for the GTC website and 3.22 for the WiC website), the information is of high
quality (3.36 for the GTC website and 3.51 for the WiC website), and the information is relevant
and up-to-date (3.20 for the GTC website and 3.31 for the WiC website). Similarly, three-quarters
(75%, 168 of 224) of online users said that information presented on the GTC-IP was easy to
understand.
47 Going to Canada Immigration Portal Qualitative Research, Phoenix Strategic Perspectives Inc., March 2007.
40
Figure 3-4: Comparison of specific attributes of quality of GTC and WiC by
intermediaries
3.51
3.36
3.31
3.22 3.20
3.13
Despite these findings and the 121 updates that have occurred on the GTC since 2008, CIC key
informants generally reported that the information on the GTC website needs to be updated
more often. The anticipated merger of the GTC with the CIC website was often cited as the
main reason for fewer updates to the GTC website. In addition, there was strong agreement
among both CIC and HRSDC key informants that the page updates on the WiC website occur
on a regular basis and those updates are sufficiently frequent. Updates to the WiC tool and the
social media pages (e.g., Facebook and Twitter) are made by HRSDC on a daily basis to ensure
they are relevant and up-to-date. CIC dashboard reports from 2010, which includes statistics on
web publishing requests and updates, also indicate that the main CIC site is updated daily.
The content provided on the provincial and territorial portals, in general, is relevant, easy to
understand and consistent with the GTC website and WiC website.
Like the GTC-IP, the quality of content on the provincial and territorial portals was reviewed by
the subject matter expert to assess relevance, clarity and currency, and also to assess the
consistency of their content with the content presented on the GTC-IP. The subject matter
expert found that the content presented on the provincial and territorial portals was both relevant
and current. That is, they provided information that is accurate and needed by new and potential
immigrants and immigrant service organizations, and the information is up-to-date, providing the
most recently available information that would help new and potential immigrants make a
decision about immigrating, living, working, settling, visiting, or studying in Canada.
Furthermore, information on the provincial and territorial portals was found by the subject
matter expert to be consistent with content presented on the GTC website and the WiC website
– a finding echoed by key informants from the provinces and territories.
In terms of the clarity of the provincial and territorial portals, the subject matter expert generally
found the content on those portals to be clear, and easy to read and understand. A few
41
suggestions were made by the subject matter expert to increase clarity on three of the provincial
and territorial portals as follows:
A number of links on the Yukon portal (French) presently link to English text, where French
text would be readily available;
Several pages on the Manitoba portal (English and French) are duplicated, but with different
URLs; and,
The Prince Edward Island portal should undergo an English editorial review.
In addition to the subject matter expert review, a number of the provincial and territorial portals
have commissioned their own language assessment studies to assess the language level (or
readability) of their portals. Language assessment studies using commonly accepted standards
and/or focus groups on readability have been commissioned to assess the language level of the
portals in British Columbia, Alberta, Ontario and Newfoundland and Labrador. All studies found
that the information presented on those portals was easy for new and potential immigrants to
understand and was presented at an acceptable level (e.g., 85% who responded to a survey posted
on the Alberta portal in 2009-10 agreed that the language was easy to understand).
A number of provincial/territorial portals have also received awards and recognition for the
overall quality of their portal. For example, the British Columbia portal received an award from
the Institute of Public Administration in Canada (IPAC) in 2009 as recognition of the portal
contributing to an improvement to a part of the public sector. The British Columbia portal also
received the BC Premier‟s Award of Excellence (2008-09), and a Public Sector Information
Technology Award (2008). More recently (December 2010), the Saskatchewan portal has won a
Silver Leaf Award from the International Association of Business Communicators (IABC) in the
Interactive Media category. In addition, several municipal websites in Ontario have won awards
for their websites as part of the Welcoming Communities‟ Initiative.
Users are satisfied with the GTC-IP and would recommend it to others.
Data from multiple lines of evidence demonstrate that the target audience is generally satisfied
with the GTC-IP. For example, focus group participants (including both users and non-users of
the Portal) said that they were satisfied with the information provided on the GTC-IP and 75%
(223 of 299) of online users reported that they were satisfied with the GTC-IP.
Online users were further asked to rate the usefulness of the information provided on
immigrating, working, settling, visiting and studying in Canada. As shown in Figure 3-5, a large
majority of respondents found the information on immigrating, settling and working in Canada
to be useful (gave a rating of 1 or 2), and at least half who were looking for information on
visiting and studying said that they found the information to be useful.
Findings presented in Figure 3-4 are supported by a 2008 usability study of the GTC-IP, which
stated that, “A majority are satisfied with all elements of the Web site included in the survey.”48
42
Figure 3-5: Percent of online users who found specific components of the GTC-IP to
be useful
100%
87% 85% 84%
80%
69%
60% 50%
40%
20%
0%
Immigrating to Settling in Canada Working in Canada Visiting Canada Studying in Canada
Canada (N=20) (N=133) (N=13) (N=16)
(N=92)
Note: The number of respondents varies by category because only those who had viewed those pages were asked to
provide an assessment of the related content.
A further measure of satisfaction is the extent to which GTC-IP target audiences would
recommend the GTC-IP to others, including friends, family members, and/or clients. A very
high percentage of online users (88%, 122 of 140) said they would recommend the GTC-IP to
others, while an additional 79% (55 of 70) of intermediaries said they would recommend the WiC
website and 77% (54 of 70) of intermediaries would recommend the GTC website to their
clients. Of those intermediaries who reported that they have recommended the WiC to their
clients, 58% (32 of 55) recommend it often, and only 7% (4 of 55) rarely recommend it. Similarly,
the majority of intermediaries who have recommended the GTC to clients, recommend it often
(61%, 33 of 54), and only 9% (5 of 54) rarely recommend it.
100%
80%
58% 61%
60%
40% 35%
30%
20%
7% 9%
0% 0%
0%
Often Sometimes Rarely Never
WiC GTC
43
The target audience is satisfied with the provincial and territorial portals.
Studies commissioned by some provinces and territories, including focus groups and satisfaction
studies/surveys, show that the target audiences are satisfied with the provincial and territorial
portals. For example, focus group testing conducted by British Columbia in 2008 with recent
immigrants and service provider organizations demonstrated that users are generally satisfied
with the information provided on the British Columbia portal. In 2009-10, 68% of respondents
who completed a feedback survey on the Alberta portal said that they were satisfied with the
website. In addition, 88% (36 of 41) of intermediaries who participated in the online survey
conducted for this evaluation said that they have recommended the provincial and territorial
portals to new and prospective immigrants.
Data from multiple lines of evidence show that the GTC-IP is providing needed information that
is helping new and prospective immigrants. As shown in Figure 3-7, sixty-nine percent (69%, 22
of 32) of online users who participated in the survey said that the information provided on the
GTC-IP was useful in helping them integrate into Canadian society. Furthermore, online users
said the information provided on the GTC-IP helped them make a decision about immigrating to
Canada (84%, 77of 92) and working in Canada (78%, 103 of 132). Online users also found the
information on the GTC-IP useful in preparing for the immigration process and for helping
them to settle in Canada. In addition to survey data, focus group participants who had used the
GTC-IP reported that the information provided on the Portal helped them prepare for the
immigration process and non-users reported that the information would have been helpful to
them had they known about it.
Figure 3-7: Percent of online user survey respondents indicating instances where
information on GTC-IP was useful/helpful
100%
84%
78% 76%
80% 74%
69%
60%
40%
20%
0%
Information useful Help make a Help make a Information useful Information useful
for helping integrate decision about decision about for preparing for for first settling
(N=32) immigrating to working in Canada immigration process (N=34)
Canada (N=132) (N=45)
(N=92)
44
Another measure of the impact of the GTC-IP is the extent to which it has resulted in increased
knowledge among new and prospective immigrants. Online respondents to the user survey were
asked if they had learned something new while visiting one of the GTC-IP sites and 77% of
respondents (229 out of 299) indicated that they had learned something new with the information
found on the Portal.
As well, intermediaries were asked to comment on the usefulness of the GTC-IP as it related to
the needs of their client-base of new and prospective immigrants. As shown in Figure 3-8, 73%
of intermediaries found the GTC-IP useful overall, including 77% (54 of 70) who said the GTC-
IP helps immigrants make informed decisions about coming to Canada, 72% (50 of 70) who said
the GTC-IP prepares prospective immigrants to work in Canada, and 77% (54 of 70) who said
the GTC-IP helps prepare prospective immigrants to immigrate to Canada.
Figure 3-8: Percent of intermediaries who found specific aspects of the GTC-IP useful
for new and prospective immigrants
100%
77% 77%
80% 72%
69% 67%
60%
40%
20%
0%
Helps make Prepares Helps newly-arrived Prepares Prepares
informed decisions prospective immigrants settle prospective prospective
about coming to immigrants to visit upon arrival immigrants to work immigrants to
Canada Canada (N=70) in Canada immigrate to
(N=70) (N=70) (N=70) Canada
(N=70)
Of those online user survey respondents who currently live in Canada but were not Canadian by
birth, 76% (34 of 45) said that they found the information on the GTC-IP to be useful in helping
them prepare to immigrate to Canada.
The move to fully integrate the Portal with the CIC main site has been favoured by CIC, with
other stakeholders and newcomers citing the potential challenges and weaknesses of providing
GTC-IP content at this location.
In September, 2008 a “Portal and CIC Main Site implementation Plan” was drafted to articulate
an approach for the partial integration of the GTC-IP into CIC‟s main site. In the Plan, it was
stated that incorporating GTC-IP content and tools into the CIC website “would maximize value
for taxpayers and CIC website visitors, namely prospective immigrants and newcomers.”
Furthermore, it stated that “as the CIC website receives approximately 50 visitors for every single
45
visitor to the GTC-IP, incorporating GTC-IP elements into the CIC website will result in a much
higher number of people exposed to [its] valuable content, without the added effort of
promoting the GTC-IP as a separate entity.”
As previously noted, the GTC-IP RMAF also provided information on what full integration of
the GTC-IP into the CIC main site could consist of. CIC, HRSDC, and provincial and territorial
key informants were all asked to provide their opinion on full integration. Responses and
opinions among those three key informant interview groups differed as follows:
a) Among the twelve CIC key informants, just over half were in favour of moving GTC-
IP content to the CIC website, five of which cited the necessary conditions that would
allow such a move, such as ensuring that GTC-IP content be placed prominently on
the CIC website. Only one CIC informant wanted the GTC-IP maintained at its
current location and URL, while three others provided pros and cons for maintaining
the URL or moving content to the CIC website. Most CIC respondents felt that a site
move would garner more visitors to the GTC-IP49. These interviewees also stated the
benefits of having all of the information in one location, which would help to reduce
duplication of content and confusion among new and prospective immigrants.
b) Among the seven HRSDC key informants, all provided responses on the pros and cons
of both options (maintaining the information in its current location versus moving the
GTC-IP to CIC); however, most were generally supportive of maintaining the GTC-IP
as a separate website at its current URL.
c) Among the eleven provincial and territorial key informants, almost half said that the
GTC-IP should be maintained as a separate website at its current URL, two were in
favour of moving GTC-IP content to the CIC website, while three provided responses
on the pros and cons of each option (maintaining the URL or moving content to the
CIC website).
A number of reasons were provided by key informants (CIC, HRSDC, and provinces/territories)
to support relocating content on the GTC-IP to the CIC website, including:
a) Having more visitor traffic to the CIC website compared to the GTC-IP50 and thus, a
greater reach to new and prospective immigrants;
b) Having all of the information in one location would reduce potential confusion among
new and prospective immigrants; and
c) Removing duplication as much of the information presented on the GTC-IP is already
located on the CIC website (e.g., the WiC tool).
Reasons provided by key informants (CIC, HRSDC, and provinces/territories) that supported
maintaining the GTC-IP at its current location as a separate website included:
a) The complexity and current layout of the CIC website would make it difficult, if not
impossible for users to find the information presently found on the GTC-IP;
b) The need to keep settlement and working information together in one location or
website; and
49 The GTC website received 53,000 unique visitors/month (January to April 2010), while the CIC website received
3,503,912 unique visitors/month (2009).
50 The WiC website received 220,000 unique visitors/month (January to April 2010), the GTC website received
53,000 unique visitors/month (January to April 2010), while the CIC website received 3,503,912 unique
visitors/month (2009).
46
c) Concerns that GTC-IP content would not be updated frequently enough if it were
moved to the CIC website.
When asked, CIC, HRSDC, and provincial and territorial key informants also said they did not
foresee any technical or branding issues that would preclude or complicate moving GTC-IP
content to the CIC website.
While focus group participants were not presented with the question of keeping versus moving
GTC-IP content to the CIC website, users of the GTC-IP indicated that the GTC-IP provided
them with information not readily available on the CIC site (e.g., information sought related to
climate, the population (culture), and day-to-day life in Canada), and that the information was
helpful to them when planning their move to Canada. Both users and non-users of the GTC-IP
generally reported that there is a need for the information provided on the GTC-IP, although
there was some ambiguity and confusion between the GTC-IP and the CIC website (particularly
among non-users who sometimes had a difficult time distinguishing between the two websites).
As further demonstrated in the focus groups (users and non-users of the GTC-IP), the CIC
website was visited most often by people looking for forms, checking on application status, or
retrieving technical documentation required to live, work, study, or visit Canada. In spite of this,
focus group participants stated that they found the layout and design of the CIC site complex,
technical and overly sophisticated in its use of the English and French languages (e.g., language
barriers).
The open dialogue and willingness to share information and tools has resulted in cost-savings
to partners who would not have the resources to develop them independently. This is a key
benefit of the Initiative.
As part of the evaluation, a comparison of the costs of the GTC-IP (e.g., costs per FTE and costs
per unique visitor) to other similar government and non-government portal initiatives was
planned. However, due to significant dissimilarities in the scope of the websites, technical
considerations and web data capture, the GTC-IP could not be compared to other portal
initiatives in terms of usage and costs. For example, each comparator website tracked different
metrics on their websites (e.g. unique visitors, page views) and used a variety of web analytics
packages to produce reports. This creates an inconsistent and incomparable value related to
website usage.
However, the evaluation did reveal that cost-savings and efficiencies resulted from participation
in the portal Initiative. The primary cost saving benefit of the GTC-IP stems from the quality
added to various sites through collaboration among Portal partners. Half of the interviewees,
who provided a response, articulated that the GTC-IP Initiative promoted the creation of
networks and partnerships and Table 3-2 in section 3.2.2 provided a list of the various ways in
which Portal partners share tools and resources. Through collaboration, the overall quality of
participant sites is enriched and more comprehensive information made available on partner sites.
In many cases, the production of tools would be cost-prohibitive if Partners were required to
develop them on their own.
Other cost-saving qualities of the Portal include the nature of the tools that have been developed
and the creative ways in which various partners have used social media to increase site traffic. For
example, as a fully embeddable source, the WiC tool is easily merged into the look and feel of any
47
site on the GTC-IP Initiative without writing additional code or further development.
Furthermore, by becoming the source for Canada‟s labour market information, the level of
outreach and exposure that the WiC tool has gained, only further acts to bring users to the GTC-
IP and lower advertising costs. HRSDC has also demonstrated effective use of Facebook,
Twitter, and YouTube, as cost-saving ways to increase the presence of the Portal.
48
Overall conclusions and recommendations
The following section provides the key findings, overall conclusions of, and recommendations
stemming from the Summative Evaluation of the Going to Canada Immigration Portal,
organized by evaluation issue.
Key Findings
Relevance
The evaluation showed that the GTC-IP Initiative is a relevant program that addresses a
continuing and demonstrated need by providing complete, up-to-date, and accurate information
to its target audience in a more unified manner than most alternative sources. Users of the GTC-
IP reported that the Portal contained the information they needed and that it increased their
knowledge of living and working in Canada. As well, they reported that the information on the
GTC-IP helped them make decisions about coming to Canada, prepare for the immigration
process, and integrate into Canadian society upon arrival.
Furthermore, the GTC-IP Initiative demonstrates a high degree of alignment with Federal
priorities related to attracting a skilled workforce, and the successful integration of newcomers.
The Federal government is well placed to provide national-level, authoritative information and to
coordinate the provision of more local-level information provided by provinces, territories, and
municipalities.
Nevertheless, a significant amount of GTC content, which was developed for the purposes of the
Portal, has been copied and incorporated into the CIC main site in order to leverage its large user
population. The lack of distinction in mandate for content provision between GTC and the CIC
main site challenges the relevance of the GTC site moving forward. The evaluation showed
uncertainty regarding future directions of the GTC-IP Initiative resulting from the pending
decision by CIC to fully integrate the GTC-IP into the CIC main site. The implications of such a
move have not been formally discussed with the partners of the Initiative, who highlighted
potential issues with integration as part of this evaluation.
Performance
The GTC-IP Initiative is a relatively new endeavour and has experienced some challenges
associated with its stage of development. The Initiative has operated under a governance model
that has not been clearly defined and implemented and does not have well-articulated decision-
making processes among partners. Key informants also reported a lack of clarity in roles and
responsibilities across partners. Governance of the Initiative within CIC has been particularly
challenging as a result of a decentralized system with four branches working independently on
overlapping program components, and an historical lack of clear leadership. The internal
governance of the WiC by HRSDC, however, consists of one team working together, allowing
for responsiveness, flexibility, creativity and innovation in WiC‟s design and delivery.
The fragmented nature of CIC‟s governance structure may have contributed to inconsistent
tracking of allocations and expenditures, including to P/Ts and across Branches. A review of
documentation also revealed that provincial and territorial reporting on expenditures was not
always completed in a timely fashion and differed in level of detail across P/Ts.
49
Despite these early challenges, recent improvements in the delivery of the GTC-IP were noted in
the evaluation, including the identification of e-Communications Branch as the CIC Portal lead,
the creation of a series of draft documents that clarify the governance of the GTC-IP, a revised
funding model within CIC that is more responsive to the needs of the Initiative, and
improvements in timeliness of reporting by provinces and territories. As well, almost all of the
P/T websites are now fully functional. It is anticipated that these improvements will continue to
add to the performance of the GTC-IP Initiative moving forward.
The primary strength of the GTC-IP Initiative has been the development of
federal/provincial/territorial/ municipal partnerships in the creation and provision of a full
spectrum of online information, from the local to the national level. Focus group participants
identified this range of information as necessary to assist their decision-making and facilitate their
integration once in Canada. The multilateral partnerships that have been created as a result of the
GTC-IP Initiative were identified as a key positive unexpected outcome. These multilateral
partnerships have resulted in the sharing of content and tools, which has created cost savings for
all partners and improved the overall quality and comprehensiveness of information available on
partner sites. The GTC-IP communications structures supporting these partnerships are effective
and the bi-annual workshops, which facilitate information sharing and relationship-building, were
highlighted as a best practice in the evaluation.
Multiple lines of evidence showed that the GTC, WiC, and P/T sites are functional, user-friendly,
relevant, easy-to-navigate and provide high quality information. As well, these sites are providing
consistent information on immigrating to and working in Canada, likely resulting from the
extensive collaboration of Portal partners. A number of provincial/territorial portals have been
the recipient of awards and received recognition for the quality of their websites. As well, the
WiC tool has shown such success that it has been positioned as Canada‟s authoritative source for
labour market information.
By comparison, the GTC site provides information for potential immigrants and newcomers
covering a wide range of topics. The evaluation showed that users of GTC are most frequently
accessing the site‟s information on immigrating to Canada and are less likely to be accessing
information on the site related to settling, studying, or visiting Canada‟s provinces and territories.
Provincial and territorial portals provide high quality information on settling and studying and
may in the best position to provide this more local level information to the target audience.
The evaluation showed that more could be done to increase the awareness of the high quality
information contained on the Portal. Promotional activities by Portal partners varied, with the
promotion of the GTC found to be limited, likely as a result of uncertainty over the permanent
location of the GTC web assets. This resulted in a lack of growth, but steady usage over time.
HRSDC‟s promotion of the WiC has been effective, facilitating growth in the profile and usage
of the WiC site over the last three years. The WiC site has also benefited from the development
of partnerships within the Initiative and the placement of the WiC tool on a number of partner
sites.
A number of provinces and territories have also engaged in promotion of their sites; however,
there is little data available (e.g., usage trends around the time of promotional campaigns) to
demonstrate the effectiveness of those campaigns, or the usage of these sites overall. A wide
range of practices in web data collection are currently employed among all partners of the GTC-
IP (including provinces and territories), which limits the assessment of ongoing performance of
the Initiative.
50
Conclusions and recommendations
The GTC-IP serves as an authoritative and comprehensive resource for potential immigrants and
newcomers to Canada, their friends and family members and immigrant-serving organizations.
However, in order for the target population to benefit most from the information housed on the
Portal, the information must be easy to find, relevant, understandable, consistent and easy to
navigate. As well, the information should fall within a clear mandate of the site that is providing
it. Therefore, if the GTC content remains on the Portal URL, it should become the authoritative
source for this information and it should be promoted in order to increase the target population‟s
awareness of it. If full integration of the GTC-IP into the CIC main site moves forward,
navigation issues and profile of the GTC-IP content must be addressed. As well, some
information currently housed on the GTC was less likely to be accessed and may be more
appropriately delivered by the provincial/territorial or municipal sites. The WiC tool is currently
the authoritative source for labour market information in Canada and HRSDC should continue
its promotion and related partnership development.
Recommendations
(CIC) Develop a strategy to address the issues associated with the location of the GTC-IP
and its related content and tools, including:
The implications if full integration of the GTC-IP into the CIC main site takes place. This
could include issues concerning navigation, organization of GTC-IP components, and
content updates, for example;
The elimination of information overlap that exists between the CIC main site and the
GTC site;
The determination of which GTC content areas are appropriate to remain on this
federally operated website.
(CIC and HRSDC) Consider ways to improve the promotion and usability of GTC-IP related
content and tools.
The multilateral partnerships established through the GTC-IP Initiative were integral in allowing
for the creation and provision of consistent, high-quality information in a cost-efficient manner.
The partnerships were identified as the key strength of this Initiative and should be maintained.
The partners of the GTC-IP should continue to develop and provide users with high quality
content and tools as they continue to have a high degree of relevance. The communication
structures used by Portal partners are also effective and should be continued, particularly the bi-
annual workshops, which were identified as a best practice. Additional multilateral
communications may improve issues with governance and reporting that were identified in the
evaluation.
Recommendations
(CIC and HRSDC) Establish a plan to clarify the horizontal governance structure of the
GTC-IP, including decision-making protocols and oversight structure.
(CIC) Develop a strategy to address issues concerning CIC internal governance.
51
(CIC and HRSDC) Consider ways for Portal partners to further facilitate multilateral
communications.
Improvements to Portal governance and communications structures would also assist in the
development of common performance measures across the Initiative, improving partners‟ ability
to assess ongoing performance. Clarity in decision-making processes and roles and
responsibilities of partners related to CIC‟s internal financial tracking system and reporting by
CIC and P/Ts would lead to benefits for the department.
Recommendations
(CIC and HRSDC) Address issues related to reporting and financial tracking:
a) (CIC) Develop a strategy to address reporting issues, both internally within CIC and
with provincial and territorial partners. This includes issues dealing with financial
tracking, the timeliness of reporting and the clarity of reporting requirements, as well as
clarity concerning the roles and responsibilities of these partners.
b) (CIC and HRSDC) Work together to establish common performance measures and
consistent web data collection across all partners of the Portal Initiative.
52
Appendix A: Going-to-Canada immigration portal logic model
Copy of the logic model (Appendix A) is available upon request to Research-Recherche@cic.gc.ca.
53
Appendix B: Evaluation matrix
Report
Evaluation Questions Indicators Potential Data Sources
Section
Relevance
1. Does the initiative continue to Degree of alignment with Departmental Strategic Outcomes Document review 3.1.2
be consistent with departmental Key informant interviews
and government-wide priorities? Degree of alignment with the Government of Canada priorities
2. Does the GTC-IP initiative Evidence of GTC-IP as information source for prospective and Web analytics (e.g. Website ranking, traffic) 3.1.1
continue to address a new immigrants Key informant interviews
demonstrated need?
Survey of prospective and new immigrants
Perceptions of program partners and stakeholders with Focus groups
respect to continued information needs. Document review
Literature review
External demand data
3. Are CIC and HRSDC the most Perception of stakeholders regarding alternative approaches Key informant interviews 3.1.3
suitable delivery mechanisms for Focus groups
these information products and Level / degree of overlap or duplication with other sources of
information (e.g. P/Ts, municipalities, FCRO) Web research
tools?
Key informant interviews (program, partners)
Performance
4. Is the Portal functional and User perceptions of functionality and user-friendliness Online user survey 3.2.6
user friendly? (taxonomy, navigation) Web analytics /Log File Analysis
External Demand Data
Usability testing
Client inquiries
IM/IT professional evaluation of quality of information
architecture / taxonomy / navigation (e.g. appropriate CLF2 compliance testing
selection and use of meta data) Program documentation
Search Engine Research
# of errors/malfunctions on the site On-line surveys
Web analytics /Log File Analysis – link analysis
(links to/from Website)
Scanning analysis of Entry Pages (i.e. pages visitors are using Help desk tickets raised
to access the Websites/Portal – Are users coming to the site at
Web analytics /Log File Analysis
the expected pages?)
Service Level Definitions
Performance of on-line service delivery (uptime and time to
fix downtime) Operations Centre Metrics
54
Report
Evaluation Questions Indicators Potential Data Sources
Section
5. Is the governance structure, Perceptions of the effectiveness of the governance structure Interviews with key informants (partners, 3.2.1
internally within each department program, management)
Including comparisons of CIC‟s GTCIP governance structure
(CIC and HRSDC), between CIC and with that of HRSDC‟s and other similar government Document review (i.e. other departmental
HRSDC, and between GoC and websites website evaluations)
P/Ts) for the Portal effective?
Senior management perceptions of governance structure
Interviews with key informants (management)
GTC-IP stakeholder and partner understanding / clarity of
roles and responsibilities Interviews with key informants (partners,
program, management)
Timeliness of decision-making
Document review
Interviews with key informants (partners,
Communication structures in place program, management)
Document review
Interviews with key informants (partners,
Adequacy of meetings to address issues and documentation to program, management)
record decisions
Document review
Key informant interviews
6. Is the Portal URL the most Stakeholder perceptions of appropriateness of location (P/Ts, Key informant interviews 3.2.10
appropriate, efficient and service organizations) document/file review (of any correspondence)
effective to meet the needs of the
Technical implications (feasibility, efficiency etc) Key informant interviews
target population?
Technical documentation
Focus groups
Branding implications
Key informant interviews (with stakeholders)
Usage patterns on Portal vs. CIC main site
Web analytics
7. Is the target population aware Extent to which Portal communications products, promotional Document review 3.2.5
of the GTC-IP? Are they using the and outreach activities are targeted/tailored to the primary Focus groups
GTC-IP? Why or why not? audience
Interviews with key informants
Web analytics
Profile and awareness of GTC-IP among prospective and new Survey of prospective and new immigrants
immigrants Key informant interviews (Stakeholders)
Usage patterns for the GTC-IP Web analytics (CIC)/ Log Files (HRSDC)
Trend analysis of referral patterns/traffic patterns (total Literature review
visits, number of pages per visit, time per visit etc)
Focus group
Usage patterns in comparison with other immigration,
settlement and labour market information sources
55
Report
Evaluation Questions Indicators Potential Data Sources
Section
Visits by date stamp compared to marketing activities Log file analysis
Key informant interviews (partners, stakeholders)
Impediments to use Focus group of users
8. Is collaboration and priority Instances of sharing of best practices and tools Document review 3.2.2
setting between and among the Key informant interviews
federal, provincial and territorial
partners effective? Evidence of information sharing between P/Ts and other Document Review
stakeholders such as municipalities, employers, immigrant
service organizations Key informant interviews (P/Ts, stakeholders)
Level of partner satisfaction regarding effectiveness of Key informant interviews (partners, stakeholders)
collaboration
Level of partner satisfaction with mechanisms for engagement Key informant interviews (partners, stakeholders)
Evidence of meetings, consultations, working groups, MOUs
and other formal agreements with partners and stakeholders Document review
9. Do the provincial/territorial Number of P/T/municipal websites developed and launched Web site review 3.2.7
portals contain and provide links to On-line user survey
relevant, up-to-date and Quality of P/T Portal content as assessed by public/service 3.2.8
providers and subject matter experts Key informant interviews (stakeholders)
understandable information on:
Focus groups
immigrating, settling, living,
visiting and studying in the Assessment by subject matter experts
User satisfaction with information provided on P/T Portal
P/Ts? On-line user survey
working in the P/Ts? Interviews (stakeholders)
Focus groups
Plain language benchmarks Public opinion research
Web analytics for links to and from portal
Visitor traffic patterns Review and ranking by a plain language expert
Web analytics / Log Files
10. Does the Portal contain and Quality of Portal content as assessed by public/service On-line user survey 3.2.7
provide links to relevant, up-to- providers and subject matter experts Key informant interviews (stakeholders)
date and understandable 3.2.8
Focus groups
information on:
Assessment by subject matter experts
immigrating, settling, living, User satisfaction with information provided on Portal On-line user survey
visiting and studying in Canada? Interviews (stakeholders)
Focus groups
working in Canada?
Public opinion research
56
Report
Evaluation Questions Indicators Potential Data Sources
Section
Average time since last page update Web analytics for links to and from portal
Review and ranking by a plain language expert
Program files
Plain language benchmarks
Web analytics /Log File Analysis
Content enhancements and releases Web analytics / Log Files
Visitor traffic patterns
11. Does the target population Users identified increased knowledge of settlement On-line user survey 3.2.9
gain knowledge and settlement information Survey of prospective and new immigrants
information regarding provinces,
Information gaps identified Key informant interviews (stakeholders)
territories and communities from
across Canada? # of users going from the GTC Website to the Focus groups
provincial/territorial Websites Provincial web analytics
# of users on pages containing settlement information Web analytics
12. Does the target population % of surveyed users identifying increased knowledge of living, On-line surveys (users) 3.2.9
gain knowledge of immigrating to working in and immigrating to Canada Survey of prospective and new immigrants.
Canada and, living and working in
Canada?
Information gaps identified Key informant interviews (stakeholders)
# of users who get a WIC report Focus groups
57
Report
Evaluation Questions Indicators Potential Data Sources
Section
15. Is delivery of the GTC-IP Extent of planned to actual resource (FTEs, O&M, Document review 3.2.3
efficient and cost-effective? contribution funds) use by planned activity
3.2.11
Ratio of management costs to funds contributed to P/Ts Document review
(management burden on contribution funds)
Cost per user (cost of initiative/# of users) and trend of that
Document review
cost
Web analytics
Cost variance with private sector options Document review (of Business Cases, MC, etc.)
Secondary research, case studies and international
comparisons
CMS cost-effectiveness and efficiency (timeliness, user
Key informant interviews
friendliness)
Document review (e.g. Close Out Reports, project
documentation)
Secondary research, case study, benchmarking
58
Appendix C: Ontario’s Municipal Immigration Information
Online (MIIO) program51
Through the Canada Ontario Immigration Agreement (COIA), and the GTC-IP initiative,
Ontario has developed a program to fund the creation of municipal immigration websites and
related online projects. Over the past five years, this has led to the launch of 21 municipal
immigration portals representing over 120 communities in Ontario.
These portals have become a model for municipal involvement and collaboration between
federal, provincial, and municipal governments with a commitment to partnership, sharing of
tools/information and delivery of innovative online projects to attract and integrate newcomers
into Ontario communities.
The MIIO program provides municipalities with the opportunity to work with community
partners to:
Deliver local immigration information;
Promote municipalities as an attractive destination for settlement;
Help newcomers integrate into the community;
Highlight local labour market gaps;
Support local employer engagement.
These municipal immigration portals are developing innovative online tools to attract and
integrate newcomers to their communities. Online mentoring tools to pair newcomers with
retirees in different labour market sectors, online trade fairs where newcomers can interact online
with potential employers, and innovative partnerships providing multilingual inquiry services
through their website. Municipal partners are also actively sharing their work with other portals,
and collaborating with federal partners to provide a wide variety of online information and
services to newcomers.
Ontario is also creating tools and content which is then offered to partners for their own use.
One example is a partnership project with the Maytree Foundation. Their HireImmigrants.ca
„Employers Roadmap‟ was adapted and embedded in different municipal partner websites,
offering employers a suite of tools to help attract, hire and retain newcomers. The tool is now
being embedded in nearly half of the municipal sites, and may also be leveraged by provincial
immigration portal partners as well.
The Municipal Immigration Information Online Program, created through COIA, has resulted in
a model for inter-governmental partnership and a network of municipal immigration websites.
These portals are a best practice in value for money, government innovation as well as a
partnership model that encourages sustainability and local engagement.
51This Appendix was submitted to the CIC Evaluation Division by the Ontario Ministry of Citizenship and
Immigration, with support from Ontario municipal partners. The MIIOs were not included in the GTC-IP
evaluation.
59
Appendix D: Websites reviewed
The following is the list of websites reviewed:
Going to Canada: www.goingtocanada.gc.ca
Se rendre au Canada : www.serendreaucanada.gc.ca
Working in Canada: www.workingincanada.gc.ca
Travailler au Canada : www.travailleraucanada.gc.ca
WelcomeBC: www.welcomebc.ca/
Immigrate to Alberta: www.albertacanada.com/immigration
Saskatchewan Immigration: www.saskimmigrationcanada.ca
Immigrate to Manitoba, Canada: www2.immigratemanitoba.com/browse
Immigrer au Manitoba (Canada) : www2.immigratemanitoba.com/browse/index.fr.html
Ontario Immigration: www.ontarioimmigration.ca/en
Immigration en Ontario : www.ontarioimmigration.ca/fr
New Brunswick: Be...in this place: www.beinthisplace.ca
Nouveau-Brunswick : Être...ici on le peut : www.etreicionlepeut.ca
Nova Scotia Immigration: www.novascotiaimmigration.com
Nova Scotia Immigration in French: www.novascotiaimmigration.com/services-en-francais
Prince Edward Island Immigration: www.gov.pe.ca/immigration
Prince Edward Island Immigration in French: www.gov.pe.ca/immigration/index.php3?lang=F
Immigrate to Newfoundland and Labrador: www.nlimmigration.ca/en.aspx
Immigrez à Terre-Neuve-et-Labrador : www.nlimmigration.ca/fr.aspx
Yukon Immigration: www.immigration.gov.yk.ca
Yukon Immigration in French: www.immigration.gov.yk.ca/fr/index.html
60
Appendix E: Documents reviewed
CIC/HRSDC and other government department documents related to priorities and
commitments:
Labour Market Information Advisory Panel - Final Report, May 2009
Comité consultatif sur l‟information sur le marché du travail - Rapport final, Mai 2009
CIC / HRSDC (see 2009-2010 RPP) - Program Activity Architecture (PAA)
CIC Letter of Understanding
CIC/HRSDC Departmental Performance Report (DPR) (2006-2009)
CIC DPR, 2008-2009 | CIC DPR, 2007-2008 | CIC DPR, 2006-2007
HRSDC DPR, 2008-2009 | HRSDC DPR, 2007-2008 | HRSDC DPR, 2006-2007
CIC/HRSDC Report on Plans and Priorities (RPP) (2007-2010)
CIC RPP, 2009-2010 | CIC RPP, 2008-2009 | CIC RPP, 2007-2008
HRSDC RPP, 2009-2010 | HRSDC RPP, 2008-2009 | HRSDC RPP, 2007-2008
Review of Recent Literature on Horizontal Management
GTC-IP program documents such as policies, briefing notes, financial reports, statistical reports,
public opinion and other research documents, usability testing reports, search engine
optimization reports, client inquiry reports, web strategy documents, etc.:
GTC-IP Project Charter
TBS Submission
GTC-Portal Evaluation Matrix
GTC-IP Terms of Reference, RFP, SOW, logic model
F/P/T Jurisdiction Template (Portal Conference 2010)
Going to Canada Website POR Quantitative Report March 2008 (IPSOS Reid)
Going to Canada Website POR Qualitative Report April 2008 (IPSOS Reid)
2007 POR WiC Tool Report
Qualitative Research on the “Working in Canada” Tool of the Going to Canada
Immigration Portal (Ekos 2007)
Executive Study - 2008 HRSDC WiC Portal Study
Final Report - 2008 HRSDC WiC Portal Study (English)
ACCC 2008 Draft Report - Going to Canada Portal
ACCC 2008 Draft Report - Compiled Comments
ACCC 2009 Report - Working in Canada Portal
April 2010 – Going To Canada Immigration Portal – Working In Canada - Final Report,
by SAGE Research Corporation.
Web Presence Audit of WiC
High-Level Social Media Guideline Review & Recommendations
Social Media Performance Measurement Framework
High-Level Microblogging Justification Document
Heuristic WiC Tool and Homepage Audit
2010 - Working in Canada Review by ACCC
61
March 2010 - Immigrants and the Internet: The information needs of Canadian immigrant
job seekers and www.workinginCanada.gc.ca by Faculty of Information - University of
Toronto.
March 2008 - Guide to Working in Canada Study, by Phoenix Strategic Perspectives Inc.
May 2009 - Working Together to Build a Better Labour Market Information System for
Canada, by Drummond
GTC-IP web-produced reports:
Portal Web metrics reports (prepared by CIC, monthly)
WiC Tool Web Metric Reports
Web Analytics/Performance Report (AWstats) April 2009
Going to Canada- Immigration Portal Statistics
GTC-IP Technical Survey
GTC-IP RMAF and Data Capacity Assessment:
GTC-IP RMAF (final version)
GTC-IP Data Capacity Analysis (Task 6)
GTC-IP Data Capacity Analysis (Task 5)
GTC-IP Data Capacity Analysis (Task 4)
GTC-IP Data Capacity Analysis (Task 3 Matrix)
Annual Service Plans of BC and MB, provincial Portal contribution agreements and
financial/activity reports, MOUs, action plans and progress reports:
Backgrounder - Funding Agreements with P/Ts
CIC Contribution Agreements with P/Ts
Alberta, Saskatchewan, Ontario, New Brunswick, Prince Edward Island, Nova Scotia,
Newfoundland and Labrador, Yukon and Northwest Territories
British Columbia and Manitoba Annual Service Plans
HRSDC MOUs with P/Ts
GTC-IP Progress Reports
GTC-IP Action Plans
Memorandum to the Minister – Request to extend existing contribution agreements with
P/Ts pertaining to the GTC-IP
Portal Workshop Updates for BC, Alberta, Saskatchewan, Manitoba, Ontario, New
Brunswick, Prince Edward Island, Nova Scotia, Newfoundland and Labrador, Yukon and
Northwest Territories
Summary and Future Agenda Items for March 2008 FPT Immigration Portal Workshop
FPT Immigration Portal Workshop Meeting Notes October 2008
FPT Immigration Portal Workshop Summary for March 2009
Provincial/territorial Status Updates March 2010
June 2010 Workshop Outcomes
Agenda and Documents for September 2010 Workshop
Communication and promotional materials, including FAQ, press releases, fact sheets:
GTC-IP Backgrounder
62
Communications Policy of the Government of Canada (TBS) – section on Internet and
Electronic Communication and Technical Innovation and New Media
Best Practices in Outreach and Digital Marketing - BC Presentation (FPT Portal
Workshop 2010)
News Release - Government of Canada supports Nunavut in attracting immigrants to the
territory
GTC-IP Promotional materials
Memorandum of Understanding between Working in Canada and their Partners:
Alberta MOU
CIC MOU
Manitoba MOU
Ontario MOU
63
Appendix F: WiC tool data matrix and process flow
Copies of the process map (Appendix F) are available upon request to Research-
Recherche@cic.gc.ca.
65