APLAC MR 001

PROCEDURES FOR ESTABLISHING AND MAINTAINING THE APLAC MUTUAL RECOGNITION ARRANGEMENT AMONG ACCREDITATION BODIES

Issue No. 17

Issue Date 06/10

Page 1 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001 PURPOSE This document describes the procedures for APLAC to follow in the evaluation and re-evaluation of an accreditation body that accredits: calibration laboratories; testing laboratories including to ISO 15189; reference material producers; and/or inspection bodies

when that accreditation body wishes to become a signatory to the APLAC Mutual Recognition Arrangement (MRA) or wishes to maintain signatory recognition in the APLAC MRA. The accreditation body may be an APLAC Full member or an accreditation body from outside of the APLAC region. This document should be read in conjunction with the following APLAC MR documents: APLAC MR 008 APLAC MRA Council Rules for Operation APLAC MR 011 A Guide for APLAC Evaluation Teams for the Planning and Conduct of Evaluations AUTHORSHIP This document is produced by the APLAC Mutual Recognition Arrangement Council. OFFICIAL LANGUAGE The text may be translated into other languages as required. The English language version is the official version. COPYRIGHT The copyright of this text is held by APLAC. APLAC publications may not be copied for sale by any individual or body other than APLAC member organisations. FURTHER INFORMATION For further information about this publication, contact the APLAC Secretariat. NATA 71-73 Flemington Road North Melbourne VIC 3051 Australia Tel: + 61 3 9329 1633 Fax: + 61 3 9326 5148 Email: aplac@nata.com.au Web site: www.aplac.org

Issue No. 17

Issue Date 06/10

Page 2 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001

TABLE OF CONTENTS Page Purpose.................................................................................................................................... 2 Authorship................................................................................................................................ 2 Official Language ..................................................................................................................... 2 Copyright.................................................................................................................................. 2 Further Information .................................................................................................................. 2 Part 1 0 1 2 3 4 5 6 General .................................................................................................................. 5

Definitions ....................................................................................................................... 5 Scope .............................................................................................................................. 6 Objective ......................................................................................................................... 6 Criteria............................................................................................................................. 7 Costs ............................................................................................................................... 8 Confidentiality and Impartiality ........................................................................................ 9 Changes to this Document.............................................................................................. 9

Part 2 7 8 9 10 11

The MRA Process............................................................................................... 10

Application..................................................................................................................... 10 Appointment of Team Leader........................................................................................ 10 Delay in the Evaluation Process .................................................................................. 10 Maintenance of the APLAC Mutual Recognition Arrangement ..................................... 10 Procedures for Extension of Accreditation Bodys Scope of Recognition under APLAC Mutual Recognition Arrangement..................................................................... 12

Part 3 12 13 14 15 16 17 18 19

The Evaluation Process..................................................................................... 12

Cooperation by the Applicant Body.............................................................................. 12 Pre-evaluation Visit ....................................................................................................... 13 Composition of Evaluation Team .................................................................................. 15 Preparation for Evaluation............................................................................................. 17 Conduct of the On-site Evaluation ................................................................................ 18 Evaluation Report.......................................................................................................... 19 Corrective Action and Evaluation Teams Recommendation to APLAC MRA Council ................................................................................................................. 20 Monitoring the Performance of Evaluators.................................................................... 21

Issue No. 17

Issue Date 06/10

Page 3 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001 Appendix A Appendix B Appendix C Appendix D Role and Responsibilities of Evaluation Team ............................................ 22 Declaration of Confidentiality and Impartiality ............................................. 23 Team Leader Evaluation Report Checklist .................................................. 24 Conditions for Provision of an Evaluation Report to Interested Parties ......................................................................................................... 25

Issue No. 17

Issue Date 06/10

Page 4 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001

PART 1 GENERAL 0 0.1 DEFINITIONS Accreditation body (AB): an authoritative body that operates an accreditation system for one or more types of conformity assessment bodies. MRA: mutual recognition arrangement. Non-member accreditation body: an accreditation body from another region or an unaffiliated accreditation body that is not a member of APLAC. Regional group: a formal group of accreditation bodies from a geographic area of the world, representing different economies, whose purpose is to develop and maintain an MRA. APLAC MRA group: the accreditation bodies that are signatories to the APLAC MRA. Accredited organisation: a calibration laboratory, testing laboratory, inspection body or reference material producer accredited by an accreditation body. Applicant body: APLAC Full Member that applies to become a signatory to the APLAC MRA or a non-member body from outside the APLAC region that applies to become a signatory to the APLAC MRA. Associate members are not eligible to apply. For a re-evaluation the phrase applicant body applies to a signatory being re-evaluated. Proficiency testing activity: for the purpose of this document, all those activities used by accreditation bodies to assess performance, including proficiency tests (refer to ISO/IEC 17043 Conformity assessment General requirements for proficiency testing) conducted by cooperations, accreditation bodies, commercial organisations or other providers. ISO/IEC Standard: an ISO/IEC standard or guide related to accreditation and/or conformity assessment. Peer Evaluation: a structured process of assessment of an accreditation body against the specified requirements. Evaluation Team Leader (TL): a person responsible for leading an MRA peer evaluation team. Evaluation Team Member (TM): a person serving on an MRA peer evaluation team. Witnessing: observation by an evaluation team of an accreditation bodys assessment of a conformity assessment bodys competence . (It may also include observing the accreditation bodys staff preparing for an assessment and dealing with assessment reports.) Nonconformity: a finding where the AB does not meet a requirement of the applicable standard(s) (e.g., ISO/IEC 17011), its own management system or the

0.2 0.3

0.4

0.5 0.6

0.7

0.8

0.9

0.10

0.11

0.12 0.13

0.14

Issue No. 17

Issue Date 06/10

Page 5 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001 MRA requirements in a way that discredits its competence or jeopardises the quality of its work. 0.15 Concern: a finding where the accreditation bodys practice may develop into a nonconformity or the team is not fully satisfied. Comment: a finding about the accreditation bodys documents or practices with a potential for improvement, although still fulfilling the requirements. SCOPE This document describes the procedures for APLAC to follow for the evaluation and re-evaluation of member accreditation bodies that apply to become signatories to the APLAC Mutual Recognition Arrangement (MRA). When used to evaluate non-APLAC members from outside the APLAC region, the same criteria and procedures in this document, other than clause 3.4 of this document and clause 3.5 of APLAC MR 008, shall be followed. The result of a successful evaluation in this case will be a contract of bilateral mutual recognition, not the signing of the APLAC MRA. OBJECTIVE The objective of an evaluation of an applicant body is to establish cross-border confidence in the endorsed reports and certificates (i.e. reports and certificates containing the accreditation bodys accreditation symbol) issued by its accredited conformity assessment bodies. The evaluation focuses on how the applicant body ensures the technical competence of its accredited conformity assessment bodies. In order to achieve this objective, the evaluation team does the following: 2.2.1 An initial appraisal of the applicant bodys documented policies and procedures as set out in its quality manual (by whatever name) and associated documentation. This appraisal is done against the relevant ISO/IEC standards, ILAC/IAF and ILAC documents (see section 3.1), and APLAC MRA supplementary requirements. An on-site evaluation of the implementation of the policies and procedures. An evaluation of the applicant bodys ability to accredit conformity assessment bodies, including an appraisal of whether the applicant body obtains sufficient evidence that the conformity assessment bodies are technically competent to perform the work for which they are seeking accreditation or have been accredited. Use of peer inspectors as an accreditation bodys technical assessors/experts is particularly important for the assessment of inspection bodies where proficiency testing and other forms of intercomparisons are not practical as a means of establishing the inspection bodys technical competence.

0.16

1 1.1

1.2

2 2.1

2.2

2.2.2 2.2.3

2.3

The evaluation process requires the exercise of some judgement and interpretation as it is unlikely that each requirement or activity will be addressed in exactly the same way in different organisations. The evaluation team shall, therefore, collect sufficient information on each topic to enable it to appraise the suitability of the practices used by

Issue No. 17

Issue Date 06/10

Page 6 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001 the applicant body. Significant differences in approach are to be highlighted in the evaluation report, as they must be made known to all signatories to the MRA. 3 3.1 CRITERIA Standards This document refers to ISO and ISO/IEC standards and guides on conformity assessment (hereinafter referred to as ISO(/IEC) standards). Unless otherwise stated, the current edition of each document applies. 3.1.1 An applicant body shall, fulfil the requirements for a single accreditation body as stated in Section 2 of ILAC/IAF A2 and Section 5 of ILAC P1, and ILAC/IAF A5. An applicant body shall ensure its accredited conformity assessment bodies fulfil the following criteria: 3.2 for calibration laboratories, ISO/IEC 17025; for testing laboratories, ISO/IEC 17025 (ISO/IEC 17025 or ISO 15189 for medical laboratories); for inspection bodies, ISO/IEC 17020; ILAC/IAF A4; for RMPs, ISO Guide 34 and ISO/IEC 17025 in combination; the mandatory clauses of APLAC TC 008.

3.1.2

Supplementary Requirements Additionally, an applicant body shall: 3.2.1 meet the requirements included in ILAC and APLAC documents decided as mandatory by the APLAC General Assembly, the list of which is available from the APLAC Secretariat. At the time of issue of this document, the current issues of the following apply: ILAC P8, ILAC P9, ILAC P10; be fully operational (i.e., have carried out surveillance and reassessment); Note 1. When an applicant body is already a signatory to the APLAC MRA and applies to extend its scope of recognition, the requirement to have completed a full cycle through to reassessment may not necessarily be applied to the requested scope extension. Each case will be considered by the APLAC MRA Council on an individual basis. 3.2.3 demonstrate that its accredited conformity assessment bodies can access an appropriate measurement system that enables them to make measurements traceable to national or international standards of measurement; for re-evaluations, abide by the requirements and obligations of the current issues of APLAC MR 001 and APLAC MR 002 (the APLAC MRA text).

3.2.2

3.2.4

Issue No. 17

Issue Date 06/10

Page 7 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001 3.2.5 An applicant body for RMP shall either already be a signatory to the APLAC MRA for calibration and/or testing, or be applying for testing and/or calibration at the same time. When the applicant applies to be a signatory for testing and/or calibration and RMP at the same time, signatory status for RMP will be granted only after signatory status has been granted for testing and/or calibration, or at the same time as signatory status for testing and/or calibration is granted.

3.3

Proficiency Testing Activity 3.3.1 An applicant body shall satisfy ILAC P9. APLAC runs proficiency testing programs and inter-laboratory comparisons (ILCs) in some fields of testing and calibration. Each applicant body or signatory to the MRA for calibration and testing shall participate, as far as available and practicable, in order to verify the technical competence of its accredited laboratories, and to demonstrate the applicant bodys ability to take corrective action when necessary. 3.3.2 Proficiency testing activities are also applicable to certain types of inspection. An applicant accreditation body to the APLAC MRA for inspection shall have a documented policy on proficiency testing, and encourage its accredited inspection bodies to participate where relevant proficiency testing activities are available. For RMP accreditation ILAC P9 applies to the relevant calibration, testing and/or measurement activities performed.

3.3.3

3.4

Non-members of APLAC Accreditation bodies from outside the APLAC region that are not members of APLAC shall enter into a contract of cooperation with APLAC before the evaluation process can be initiated.

4 4.1

COSTS The applicant body shall pay the expenses for each member of the evaluation team engaged in travel related to the evaluation, including accommodation and all transportation. Any other costs shall be paid by the organisations providing the evaluators. The applicant body may, however, pay for meals or parts of the meals, within reasonable limitations. The evaluation team may consist of evaluators and technical experts. When an evaluator or technical expert who comes from outside the APLAC region is included on the team, the APLAC Board of Management shall determine whether APLAC itself should pay for inter-country travel costs. Transportation expenses include airfares, taxi fares, costs for the use of privately owned vehicles, train fares, or the costs associated with other means of travel for both inter-country travel and travel within the economy. The applicant body shall pay expenses for pre-evaluations, evaluations (including multi-part evaluation visits), follow-up evaluations, surveillances and re-evaluations,

Issue No. 17

Issue Date 06/10

Page 8 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001 and for travel associated with witnessing of on-site assessments done by the applicant body. If the applicant body requests that the evaluation emphasise certain areas of its accreditation activities (see clause 15.2), and such emphasis requires extra evaluation activities, the applicant body shall bear all the extra costs involved. Extra costs would typically include the cost of the extra time of the evaluators for the evaluation, report and follow-up, and additional accommodation, subsistence and travelling expenses, and related sundry costs. 4.2 4.3 The evaluators time is donated at no charge to the applicant body. An estimate of inter-country travel expenses and details of invoicing procedures should be agreed in advance between the applicant body and the individual evaluators. The applicant body shall pay for economy class airfare unless other arrangements are agreed in advance. 4.4 4.5 Observers shall pay all their own costs. Evaluators and the applicant body shall cooperate by timely submission of invoices and timely reimbursement. CONFIDENTIALITY AND IMPARTIALITY All information received, both in writing and by spoken word, during evaluations and re-evaluations shall be treated as confidential by all parties and persons concerned. This includes information relating to both the applicant body and the accredited or applicant conformity assessment bodies visited. The applicant body and evaluation team shall agree in writing on the storage or safe disposal of documents that have been provided to the evaluation team. A copy of this agreement shall be sent to the APLAC Secretariat for retention on file. Members of evaluation teams shall sign the declaration of confidentiality and impartiality form given in Appendix B. Those observers who observe all parts of the evaluation, including the closed sessions of the evaluation team, shall also sign this form. The applicant body is responsible for the confidentiality requirements placed upon any observers who observe only the open sessions of the evaluation, i.e. those where the team is interacting with the accreditation body. The signed form shall be attached to the final evaluation report. The Chair of the MRA Council shall ensure that all applications for MRA signatory status are handled in an expeditious and non-discriminatory manner, and that the evaluation process is implemented in accordance with Sections 3, 4 and 5 of APLAC MR 008, and Part 3 of this document. CHANGES TO THIS DOCUMENT This document may be revised by the APLAC MRA Council with the agreement of the APLAC General Assembly.

5 5.1

5.2

5.3

5.4

6 6.1

Issue No. 17

Issue Date 06/10

Page 9 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001 6.2 Unless otherwise stated, any changes to this document apply from the date of issue. If a different implementation date applies, this date shall be stated in the amended document.

PART 2 THE MRA PROCESS 7 APPLICATION Within the APLAC membership, applications can be lodged only by APLAC Full members, using APLAC MR 003. An Associate member is not eligible to apply.

EVALUATION The evaluation team leader is delegated authority by the MRA Council to make final decisions regarding the conduct of the evaluation. (S)he shall have ultimate responsibility for all phases of the evaluation, including: (a) (b) (c) (d) (e) (f) composition of the evaluation team document review planning the evaluation visit conducting the evaluation visit planning and conducting any follow-up activities, including an on-site follow-up visit, reporting the results of the evaluation

The evaluation team leader shall organise and conduct the evaluation in accordance with the procedures set out in Part 3 of this document. After completing the process, the team leader shall send the evaluation report, the applicant bodys corrective action and response report (accompanied by all supporting documentation), the teams recommendation, and a list of the names of the conformity assessment bodies whose assessments were witnessed (four-part package) to the APLAC Secretariat and to the MRA Council Chair, both electronically and as an unbound hard copy. The list of CABs shall be kept on file for future reference, and shall not be sent to the MRA Council with the other information used for decision-making. The supporting documentation shall not be sent to the MRA Council but a copy shall be available at the MRA Council meeting for review if necessary 9 DELAY IN THE EVALUATION PROCESS Please see APLAC MR 008, section 5. 10 MAINTENANCE OF THE APLAC MUTUAL RECOGNITION ARRANGEMENT

10.1 Notification of Change 10.1.1 As required by the APLAC MRA, each signatory shall ensure that all of the other signatories are informed, in writing, of any significant changes in status and/or operating practices. The Chair of the MRA Council and APLAC Secretariat shall also be informed. The following information shall be provided as soon as possible after the change(s), together with the accreditation bodys own analysis of how, and to what extent, the change impacts on its technical competence and APLAC MRA signatory status, etc:

Issue No. 17

Issue Date 06/10

Page 10 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001

(a)

details of any change in the name or legal or corporate status of the accreditation body or its parent organisation; details of new mutual recognition arrangements negotiated with other accreditation bodies or other parties, and of the revision, suspension or termination of any existing mutual recognition arrangements; details of any significant changes in key senior staff and the organisational structure of the accreditation body or its parent organisation; significant changes in the mode of operation of the system and in particular in the mechanisms used to assess organisations; details of the use of a sub-contracted organisation(s) to do assessments, either routinely or from time-to-time.

(b)

(c)

(d)

(e)

10.1.2

When a notification of such changes is received, the APLAC Secretariat shall inform the APLAC MRA Council Chair of the changes, who shall then decide if further information needs to be provided by the accreditation body. The APLAC MRA Council Chair may appoint an ad hoc review team consisting of one or more experienced evaluators, one of whom is preferably the evaluation team leader for the most recent on-site evaluation of the accreditation body, to review the impact of the changes on conformity of the accreditation body with the relevant MRA requirements. If the previous team leader is not able to be part of the review team, at least one member of the previous evaluation team should be included in the review team. The APLAC Secretariat shall provide the notification of changes and any additional information to the review team. The review team shall review and evaluate the information given and shall have authority to request more information, as necessary, directly from the accreditation body. Any additional information provided shall be copied to the APLAC Secretariat for inclusion in the accreditation bodys file. The review team shall also have the authority to recommend to the APLAC MRA Council (via the Chair) that it is necessary to conduct an on-site visit to the accreditation body. All costs shall be met in accordance with section 4 of this document. The review team shall prepare a recommendation to the APLAC MRA Council as to the impact of the change on the MRA status of the accreditation body in question, for discussion and decision, if necessary, at the next APLAC MRA Council meeting.

10.1.3

10.1.4

If the changes notified by the accreditation body are significant or if the review (10.1.3) finds significant nonconformities, the APLAC MRA Council may consider the need to suspend or withdraw the accreditation bodys MRA signatory status.

Issue No. 17

Issue Date 06/10

Page 11 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001 10.2 Monitoring and Re-evaluation of MRA Signatories 10.2.1 Each signatory to the APLAC MRA shall be re-evaluated at a maximum interval of four years from the first day of the final full evaluation visit of the initial evaluation of the accreditation body. If a signatory has not been re-evaluated after four years, the subsequent meeting of the MRA Council may discuss the need for suspension of the MRA signatory status of the accreditation body. A re-evaluation shall take place at a shorter interval should there be due cause, such as notification of significant changes (see clause 10.1.1). An evaluation for extension of MRA signatory status may be conducted, together with the next re-evaluation if the accreditation body applies for an extension to the scope of recognition. A re-evaluation visit shall be done by a team chosen in accordance with the procedures described in part 3 of this document, and equivalent evaluation procedures shall be used for a re-evaluation as were used for the initial evaluation. The re-evaluation should concentrate on examining changes at the applicant body and in its documentation, on compliance with the current issues of the relevant ISO(/IEC) standards (especially when a new edition of the standard has been issued since the previous evaluation) and any other new supplementary MRA requirements adopted by APLAC and ILAC, and on obtaining evidence that accredited conformity assessment bodies continue to operate in compliance with the relevant ISO/(IEC) standards.

10.2.2

10.2.3

11

PROCEDURES FOR EXTENSION OF AN ACCREDITATION BODYS SCOPE OF RECOGNITION UNDER THE APLAC MRA If a signatory to the APLAC MRA wishes to extend its scope of recognition (testing, ISO 15189, calibration, inspection and/or RMP) in the MRA, the same procedures described in parts 2 and 3 of this document shall apply, in general. For an evaluation carried out solely for the extension of recognition, only those aspects of the accreditation body relating to its accreditation activities covered by the proposed extension need to be covered. The size of the evaluation team shall correspond to the evaluation activities to be carried out.

PART 3 THE EVALUATION PROCESS 12 12.1 COOPERATION BY APPLICANT BODY The evaluation team leader shall endeavour to solicit that cooperation from the applicant body that is necessary for the conduct of an effective evaluation. The applicant body shall cooperate with the evaluation team fully and without delay throughout the evaluation process. Any unavoidable situation that could lead to a delay shall be advised by the applicant body to the team leader as soon as possible. Cooperation shall include but not be limited to:

12.2

Issue No. 17

Issue Date 06/10

Page 12 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001 (a) provision in a timely manner of all documentation and relevant information necessary for the evaluation (see APLAC MR 003, appendix 2) making necessary arrangements for the evaluation visit, that include: (i) ensuring that key personnel, staff members, assessors and committee members of the applicant body are available for interview providing the evaluation team with a list of assessments that are scheduled to take place from about 6 weeks before the proposed on-site evaluation date organising for the witnessing of a suitable number of assessments and, if applicable, for other technical visits, in conjunction with the evaluation team, and with the agreement of the conformity assessment bodies to be assessed arranging accommodation and transportation for the witnessing of assessments and, if necessary, for a visit to the national measurement institute providing the opportunity to attend a meeting of the committee concerned with decisions on accreditation, if such a committee exists and is due to meet during the visit providing meeting and working space for the evaluation team, access to a personal computer and to a photocopier, and telecommunication facilities between the team members if they will be separated by long distances during the evaluation providing interpreters, if necessary.

(b)

(ii)

(iii)

(iv)

(v)

(vi)

(vii)

Note 1. English is the official language for APLAC evaluations and the applicant body is only required to provide translation and/or interpretation from its native language into English. It is not required to provide translation or interpretation into a third language. (c) (d) timely payment of all costs as specified in section 4 of this document making an effort to build consensus with the evaluation team on the findings given in the evaluation report undertaking appropriate cause analysis and/or corrective actions on nonconformities and concerns raised in the evaluation report.

(e)

13 13.1

PRE-EVALUATION VISIT A pre-evaluation visit is optional for the applicant body. Findings from the pre-evaluation visit shall be used to determine whether the applicant body is ready for a full evaluation. A pre-evaluation visit shall not pre-empt the evaluation. The pre-evaluation visit shall be conducted by the pre-evaluation team leader and a team member chosen by the team leader, in consultation with the Chair of the MRA

13.2

Issue No. 17

Issue Date 06/10

Page 13 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001 Council, from the evaluator competencies list maintained by the APLAC Secretariat on behalf of the MRA Council. Note 1: A team leader should avoid using another lead evaluator as a team member. 13.3 After examination of the documentation by the pre-evaluation team, the team leader shall advise the applicant body of the intended agenda for the pre-evaluation visit, and seek an assurance that key personnel will be available during the visit. A provisional date for the pre-evaluation visit shall be agreed, subject to supply of the required documentation at least three months prior to the visit. During the pre-evaluation visit, the pre-evaluation team leader shall discuss with the head of the applicant body its participation in APLAC and other regional and international accreditation activities. (a) After receipt and evaluation of the documentation the pre-evaluation team shall visit the applicant body. During the visit the team shall discuss the quality system, quality documentation and their implementation, and make recommendations, where necessary, on actions to be taken before a full evaluation could be done. The pre-evaluation team shall also indicate how many days the full evaluation is likely to take. During the visit, the pre-evaluation team shall provide all necessary information about APLAC MRA requirements and about the full evaluation process. In the case of a calibration laboratory accreditation body, a part of the pre-evaluation shall be an assessment of the existence of laboratories providing traceability to the highest level in the economy or region. This is especially necessary where the traceability schedules are not clear and where participation in CIPM activities by the domestic national measurement institute is not fully known. Participation in international proficiency testing activities should also be covered. During the pre-evaluation visit, the pre-evaluation team may also visit one or two accredited conformity assessment bodies to gain an initial impression of the operation of the accreditation system and of the technical competence of the accredited conformity assessment bodies. Such a visit should not, however, to be represented as the witnessing of an assessment. A pre-evaluation visit should, in most cases, take 2 to 3 days. The duration may be varied by agreement between the pre-evaluation team leader and the applicant body. At the end of the pre-evaluation visit, the pre-evaluation team leader shall submit a short written report to the applicant body and to the Chair of the MRA Council with a copy being sent to the APLAC Secretariat. The report shall, as a minimum, contain the following information: (i) main nonconformities found, referenced to the relevant clauses of ISO/IEC 17011 and/or other APLAC MRA criteria documents;

13.4

13.5

13.6

(b)

(c)

(d)

(e)

13.7

(a)

(b)

Issue No. 17

Issue Date 06/10

Page 14 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001 (ii) (iii) the degree to which the applicant body fulfils the relevant criteria; a recommendation on whether to continue, to suspend or to withdraw the evaluation process; a recommendation on the type and number of team members necessary, and the estimated duration of any proposed evaluation visit; the conditions to be fulfilled before the evaluation visit is conducted.

(iv)

(v) (c)

The applicant body shall be given an opportunity to comment on any factual errors in the report. On the basis of the report, the applicant body shall provide the pre-evaluation team leader with a corrective action report, detailing the actions taken on the nonconformities and providing any supporting documentation. This report shall be provided within three months of the receipt by the applicant body of the pre-evaluation report. Note 1. The pre-evaluation team leader should advise the applicant body, within one month of receiving the response, whether or not the corrective actions are acceptable. Note 2. The applicant bodys response should be able to be inserted as text against each finding that is presented in a table format in the pre-evaluation report.

(d)

(e)

Once the pre-evaluation team is satisfied with the response from the applicant body to the nonconformities the pre-evaluation team leader shall, in consultation with the Chair of the APLAC MRA Council (or the APLAC Chair if the team leader is the Chair of the MRA Council), make a decision to proceed with the initial evaluation. Neither the applicant body nor any other parties shall use the pre-evaluation report to claim that the applicant body has been evaluated by APLAC. The applicant body may appeal in writing to the Chair of the MRA Council against any decision to suspend or withdraw the evaluation. The MRA Council, during a meeting or by correspondence, shall rule on whether to uphold the appeal and to thus proceed with the full evaluation.

(f)

(g)

14 14.1

COMPOSITION OF EVALUATION TEAM For a full evaluation visit, the team leader shall choose the members of the team from the evaluator competencies list, prepared and kept up-to-date by the APLAC Secretariat on behalf of the MRA Council, in consultation with the Chair of the MRA Council and/or a sub-committee, needed to cover the technical fields, size and complexity of the applicant body. Note 1: A team leader shall avoid using another lead evaluator as a team member.

14.2

No team member shall be associated with any accreditation body that has provided a consultancy service to the body being evaluated during the last four years.

Issue No. 17

Issue Date 06/10

Page 15 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001

14.3

The team chosen shall consist of representatives from a cross-section of APLAC member accreditation bodies and, if appropriate, representatives of other recognised contract of cooperation partners or ILAC-recognised regional groups. The inclusion of more than one team member from a single economy should be avoided. Evaluators designated by the International Laboratory Accreditation Cooperation (ILAC) or from other non-APLAC member bodies or regional groups recognised by ILAC may also be accepted to carry out evaluations. The team leader should also take into account of the following when choosing the members of the team. (a) The team leader should avoid appointing the same team for two successive evaluations of the same applicant body. For the purposes of team composition, a pre-evaluation visit prior to an evaluation visit and a follow-up evaluation visit after an evaluation visit are treated as part of the same evaluation. For calibration and testing laboratory accreditation bodies, at least one team member shall be familiar with the application of proficiency testing as an assessment tool. For RMP accreditation bodies, at least one member of the team shall have appropriate expertise. Technical experts may be specially invited for a specific evaluation or re-evaluation. The team leader is responsible for choosing technical experts, in consultation with the APLAC MRA Council Chair. One or two observers may be added to the team with the prior agreement of the applicant body. Note 1. An observer is usually a provisional evaluator, a representative from another region, a regulator from the applicants economy or a regulator from a third economy. (e) Knowledge of the local language should be taken into account when assembling the team.

14.4

14.5

(b)

(c)

(d)

14.6

Qualification requirements for evaluators and lead evaluators are given in APLAC MR 004. The applicant body shall be informed of the names of the team members nominated to carry out the evaluation, with sufficient notice so that the applicant body has the opportunity to appeal against the appointment of any team member. The applicant body shall also be informed of the name of any observers. Once the team members have been finalised the team leader shall promptly advise the APLAC Secretariat of the names of the team members and their specific assigned tasks, and shall provide written evidence that the applicant body has accepted the team members. The Secretariat shall then issue letters of appointment to the team members.

14.7

14.8

Once the evaluation team has been accepted by the applicant body, all significant communication (other than about logistical arrangements) between the evaluation

Issue No. 17

Issue Date 06/10

Page 16 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001 team and APLAC shall be conducted through the MRA Council Chair. A copy of all correspondence shall be sent to the APLAC Secretariat for inclusion in the official files.

15 15.1

PREPARATION FOR EVALUATION The team leader shall organise the full evaluation. If a pre-evaluation has taken place, the full evaluation visit shall not be carried out until the applicant body has undertaken all the actions agreed after the pre-evaluation visit. The applicant body may request that the evaluation emphasise a certain area(s) of its accreditation activities, such as EMC to domestic and foreign regulations. In such a case, organisation of the evaluation team and the mode of the evaluation process may need special consideration so that the request can be met. The evaluation report should detail the accreditation bodys capabilities in the specific area(s) identified, and the MRA Council may decide to include reference to this specific area of recognition within the applicant bodys scope of recognition listed in APLAC MR 002. Such service shall only be provided on condition that the normal evaluation activities will not be adversely affected.

15.2

15.3

The team members shall be allocated specific tasks by the team leader prior to the evaluation. The evaluation team shall conduct the document review. The full evaluation visit shall not be carried out before it appears, from the documentation supplied by the applicant body, that it meets the criteria. 15.4.1 The team leader shall ask the Chair of the APLAC Proficiency Testing Committee to provide a brief summary report on the participation and performance in APLAC PT programs by the applicant bodys accredited laboratories prior to the evaluation. The evaluation team shall analyse the information provided. The team leader shall ask the APLAC Secretariat to provide details of the applicant bodys voting history in APLAC postal ballots. A report on this voting history shall be included in section 5 of the evaluation report (APLAC MR 009). The following aspects shall be taken into account when planning the evaluation: (i) the availability of traceability of measurements to national and international standards of measurement; the pre-evaluation report (if a pre-evaluation has been done); any evaluation experience of individual APLAC member bodies with the applicant body; information from reports from previous evaluations or re-evaluations.

15.4

15.4.2

15.4.3

(ii) (iii)

(iv)

Issue No. 17

Issue Date 06/10

Page 17 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001 15.5 The team leader (in consultation with the team members, when necessary) and the applicant body together shall decide upon the agenda for the evaluation visit, taking into account the scope of the accreditations offered and the time needed to conduct an effective evaluation. The evaluation team shall make provision in the visit agenda for time to prepare a summary report and a draft of the final report (see APLAC MR 009). The team leader shall obtain confirmation of the arrangements listed in 12.2 (b) from the applicant body. CONDUCT OF THE ON-SITE EVALUATION Team Meetings The team shall hold a meeting in private to consult about the aspects to be evaluated by each team member before commencing the on-site evaluation. The team shall allow sufficient time to discuss its findings in private at the end of each day or session, and should seek clarification arising from observations of on-site assessment activities before formulating its findings. 16.2 Opening Meeting An opening meeting shall be held with the senior management of the applicant body to confirm the objectives of the visit, the criteria to be used, the visit agenda, and the arrangements for reporting the observations arising from the on-site visit. After this meeting, the team will usually split up so that each member can do those evaluation tasks assigned to her/him. 16.3 On-site Evaluation The evaluation team shall do an evaluation of the operational procedures and practices of the applicant body at its offices, and at conformity assessment bodies undergoing assessment/re-assessment and surveillance, placing emphasis on the issues described in section 2 of APLAC MR 011. In addition, the team shall take account of the overall performance of the accreditation body. In this regard, the team shall take into consideration the ILAC/IAF A3 Key Performance Indicators (KPIs) when evaluating the accreditation bodys performance and when preparing the evaluation report (see APLAC MR 009). KPIs shall not, however, be used as additional evaluation requirements. Note 1. Witnessing of assessment activities should normally be made after a preliminary meeting with relevant members of the staff of the applicant body, and after any queries about the operational procedures and technical requirements of the applicant body have been answered.

15.6

.15.7

16 16.1

16.4

Closing meeting 16.4.1 A closing meeting shall be held with the senior management of the applicant body to present the findings from the on-site evaluation.

Issue No. 17

Issue Date 06/10

Page 18 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001

16.4.2

The team shall present the applicant body with a short summary (see Section 1 of APLAC MR 009) identifying strengths, weaknesses and conclusions, and signed by all team members at the closing meeting. A list of nonconformities, concerns and comments shall be attached to the summary report. The team leader shall give the applicant body an opportunity to comment on and discuss the nonconformities, concerns, comments, and the teams conclusions, and to clear up any misunderstandings that may have arisen. The evaluation team shall allow time in the visit agenda to prepare a draft of the final report (see APLAC MR 009). This draft shall be based on observations made and on other factual information, agreed by the team during the evaluation, including observations made and any clarifications provided by the applicant body. Note 1. The team leader should also present a more detailed oral summary of the content of the draft final report to the applicant body at the closing meeting.

16.4.3

16.4.4

The team should also determine the method of follow-up for all nonconformities and concerns identified, including any follow-up visit, if applicable, with the agreement of the applicant body. Approval by the MRA Council is not required for any follow-up activities, including on-site visits, before the final report and recommendation of the team are presented to the MRA Council. If any problems or difficulties are encountered during the course of the evaluation, including for only part of the scope, possible options available to the team leader are to withdraw the team from the visit, to change the visit to a pre-evaluation visit (for an initial evaluation only) or to treat the visit as an incomplete evaluation that requires a further visit. The preferred option is to treat the visit as an incomplete visit. A proposal to withdraw from the visit or to change its purpose shall be discussed by the team leader with the Chair of the MRA Council, unless impracticable, before any such decision is made.

16.4.5

17 17.1

EVALUATION REPORT After the visit, the team leader shall complete the report and, subject to the approval of the final draft by the team members, provide it to the applicant body, within two months. The report shall be in the format of APLAC MR 009, the evaluation report template. Note 1. The report should clearly highlight compliance with the requirements of the relevant ISO(/IEC) standard(s), APLAC MRA supplementary requirements, when relevant, and the applicant bodys own requirements.

17.2

When the team leader is not able to complete the evaluation report and report the findings of the evaluation team within the time prescribed by the MRA Council, (s)he shall advise the MRA Council Chair of the situation. The MRA Council shall re-arrange the schedule or shall appoint a new team leader to complete the task. When a new team leader is appointed, the previous team leader shall send the new team leader all information gathered to date in relation to the evaluation.

Issue No. 17

Issue Date 06/10

Page 19 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001

17.3

The applicant body shall be given the opportunity to correct any misunderstandings or errors of fact appearing in the report. Under the conditions detailed in Appendix D, the APLAC MRA Council may permit an evaluated accreditation body to provide copies of its evaluation report to interested parties. Compliance with Appendix D shall be checked by the evaluation team at each evaluation, and details shall be included in the evaluation report. CORRECTIVE ACTION AND EVALUATION TEAMS RECOMMENDATION TO APLAC MRA COUNCIL Corrective Action The applicant body shall provide the team leader with a response that consists of the corrective actions taken on the nonconformities, and responses to concerns raised in the evaluation report, together with all relevant supporting documentation. Where relevant, the report shall include the time schedule to address the nonconformities. For an initial evaluation this report shall be provided within three months of receipt of the evaluation report. For a re-evaluation this report shall be provided within one month of receipt of the evaluation report. Note 1. The team leader should advise the applicant body within one month of receiving the response whether the corrective actions are acceptable, Note 2. Preferably, the accreditation bodys response should be inserted as text against each finding presented in table format in the report, with attachments of supporting evidence of corrective action as appropriate.

17.4

18

18.1

18.2

Evaluation Teams Recommendation to APLAC MRA Council 18.2.1 After a satisfactory response by the applicant body to the nonconformities and concerns raised in the report on the evaluation, and following the findings of any follow-up visit (if applicable), the team leader shall discuss with the other members of the evaluation team the recommendations to be made to the APLAC MRA Council. The recommendations for an initial evaluation shall include: (a) whether or not the applicant body should be accepted as a signatory to the APLAC MRA the scope of signatory recognition within the APLAC MRA the timeframe for any follow-up visit and/or full re-evaluation.

(b) (c)

For a re-evaluation the recommendations shall include: (a) whether or not recognition as a signatory to the APLAC MRA should be continued any variation to the scope of signatory recognition the timeframe for any follow-up visit and/or full re-evaluation.

(b) (c)

Issue No. 17

Issue Date 06/10

Page 20 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001

If the team recommends that the applicant body not be accepted as a signatory to the APLAC MRA, or that signatory status not be continued after a re-evaluation, the report shall include the reasons for this recommendation. 18.2.2 During the process of 18.1 and 18.2.1, the team leader shall consult her/his team members as necessary. If the corrective action is not acceptable, the team leader should discuss it with the applicant body. The team leader shall send to the APLAC Secretariat: (1) (2) the evaluation report using the report format, APLAC MR 009 the applicant bodys corrective action and response report (accompanied by all supporting documentation) and the teams evaluation of them, including findings from any follow-up visits the teams recommendation (consisting of a letter detailing the evaluation teams recommendation to the APLAC MRA Council) a list of the names of the conformity assessment bodies whose assessments were witnessed as part of the evaluation. This four-part package shall be sent electronically (as Word documents) to the APLAC secretariat at least one month prior to the MRA Council meeting. A completed checklist (see Appendix C) shall be sent with the electronic package. Signed originals of the report, etc. shall be sent in hard copy separately to the Secretariat. 18.3 Presentation of Evaluation Report at APLAC MRA Council Meeting The team leader for the evaluation shall attend the APLAC MRA Council meeting at which the report on the evaluation that s(he) led is considered. If the person does not routinely attend MRA Council meetings, APLAC shall provide funding for her/his attendance. 19 MONITORING THE PERFORMANCE OF EVALUATORS The team leader and team members shall follow the procedures of APLAC MR 004 to evaluate the performance of the team leader and team members, using Appendix 3 or Appendix 4 as appropriate.

18.2.3

(3)

(4)

Issue No. 17

Issue Date 06/10

Page 21 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001

APPENDIX A ROLE AND RESPONSIBILITIES OF EVALUATION TEAM The role and responsibilities of the evaluation team are as follows. A1. The evaluation team shall evaluate compliance of the applicant body with the APLAC MRA criteria stated in section 3 of this document. The two other key tasks of an MRA evaluation team are to: (a) evaluate the effectiveness of the applicant bodys assessment team by observing: (i) (ii) (iii) whether the applicant bodys requirements are implemented whether the applicant bodys procedures for assessment are followed whether the requirements of the relevant ISO(/IEC) standard(s) are implemented satisfactorily by accredited conformity assessment bodies

(b) verify whether the technical competence of the accredited conformity assessment body is appropriate to its accreditation scope of accreditation. A2. The evaluation team leader shall, in addition to being responsibile for managing the evaluation, mentor any "provisional evaluator" assigned to the team. Mentoring includes allocating her/him such tasks as s(he) is capable of performing, supervising, and providing a report to the MRA Council Evaluator Performance Working Group about the performance of the provisional evaluator. Evaluation team members shall cooperate with the team leader and other team members and treat one another with mutual respect. The evaluation team should strive to reach consensus, and any disagreement should be resolved amicably.

A3.

Issue No. 17

Issue Date 06/10

Page 22 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001 APPENDIX B DECLARATION OF CONFIDENTIALITY AND IMPARTIALITY This form shall be signed by all members of the APLAC peer evaluation team and by all observers who observe all parts of the evaluation, including the closed sessions of the evaluation team. The accreditation body being evaluated shall be responsible for the confidentiality requirements laid on observers who observe only the open parts of the evaluation, i.e. when the evaluation team is interacting with the accreditation body. I declare that I will not reveal confidential information gained through peer evaluations or peer evaluation reports to anyone who does not have the right of access to such information and who has not signed an APLAC Confidentiality Declaration Form. After the conclusion of the evaluation I will dispose of all documents confidentially as agreed with this accreditation body either prior to or during the evaluation. I further declare that I have no conflicts of interest with this accreditation body that would compromise impartiality, and that I have not undertaken consultancies to it during the last four years.

Date: Name Affiliation Role in Evaluation Signature

Issue No. 17

Issue Date 06/10

Page 23 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001 APPENDIX C

APLAC MRA EVALUATION TEAM LEADER REPORT CHECKLIST

The following information for each evaluation and follow-up visit must be sent by the team leader to the APLAC secretariat electronically (as Word documents). Signed originals are to be sent in hard copy (refer to APLAC MR 001, clause 18.2.3):. 1. 2. 3. final report (using the report template MR 009) a letter detailing the evaluation teams recommendation to the APLAC MRA Council the accreditation bodys corrective action and response report (corrective actions to nonconformities, responses to concerns), including any supporting information provided a list of the names of the CABs whose assessments were witnessed as part of the evaluation

4.

Accreditation Body: Team Leader Eval Date Final Report * Letter of Rec. * AB Response *

List of CABs

* note date sent to secretariat

Issue No. 17

Issue Date 06/10

Page 24 of 25

Establishing and Maintaining Mutual Recognition Arrangements Amongst Accreditation Bodies - APLAC MR 001 APPENDIX D CONDITIONS FOR PROVISION OF EVALUATION REPORT TO INTERESTED PARTIES A report on the evaluation of an accreditation body, carried out on behalf of the APLAC MRA Council, shall not be published in the public domain. An accreditation body may, however, choose to make the full report available to its interested parties under the conditions detailed in points 1 to 4 below. Adherence to these conditions for provision of the evaluation report to interested parties will be examined at each evaluation and the evaluation team will report on that adherence in its report. 1. The evaluation report shall not be made available until after it has been formally considered by the APLAC MRA Council. The full evaluation report (including the accreditation bodys responses to the findings) and the MRA Council resolution arising from the consideration of the report shall be provided collectively. The report and resolution (point 2 above) shall be provided to individuals or individual organisations with an appropriate statement as to the confidential nature of the information, i.e. the information shall remain confidential to the accreditation body and the recipient. Where there is a joint evaluation between regions, the evaluation report shall not be provided to any interested party unless there is agreement by all the relevant regions and the accreditation body.

2.

3.

4.

Issue No. 17

Issue Date 06/10

Page 25 of 25