Docket: 1828 Saskatchewan Government v. General Employees Union, 2011 SKCA 52

THE COURT OF APPEAL FOR SASKATCHEWAN Citation: 2011 SKCA 52 Date: 20110502 Between: Docket: 1828 Saskatchewan Government and General Employees Union Appellant (Respondent) - and - Saskatchewan Institute of Applied Science and Technology Respondent (Applicant) Coram: Lane, Richards and Ottenbreit JJ.A. Counsel: A. Richard Engel, Q.C. for the Appellant Robert J. Gibbings, Q.C. and Deidre L. Aldcorn for the Respondent

Appeal: From: 2009 SKQB 303 Heard: October 19, 2010 Disposition: Appeal allowed Written Reasons: May 2, 2011 By: The Honourable Mr. Justice Richards In Concurrence: The Honourable Mr. Justice Lane The Honourable Mr. Justice Ottenbreit Richards J.A. I. Introduction [ 1 ] The Appellant, Saskatchewan Government and General Employees Union (“SGEU”) represents employees of the Respondent, Saskatchewan Institute of Applied Science and Technology (“SIAST”). [ 2 ] Two of SGEU’s members were interviewed for a job in Prince Albert. Both were unsuccessful and, as a result, they filed grievances.

An arbitrator found in their favour, concluding that the interview process had been flawed in several respects. SIAST then brought a judicial review application in the Court of Queen’s Bench. The Chambers judge set aside the Arbitrator’s award on the basis it was unreasonable. SGEU now appeals from the Queen’s Bench ruling. [ 3 ] In order to resolve this appeal, it is necessary to examine both the relevant terms of the Collective Agreement and the details of how the interview panel went about its business.

As explained below, I conclude that, although the Arbitrator made a number of reviewable errors in her analysis, she nonetheless acted properly in upholding the grievances. SGEU’s appeal must be allowed. II. Basic Background [ 4 ] SIAST delivers adult educational programs. It employs various instructors, program heads and consultants. [ 5 ] In March of 2008, SIAST posted a Continuing Educational Consultant position in the Industrial Training Division of its Prince Albert campus.

The position involved the design, development and delivery of programs in consultation with business and industry. [ 6 ] The posting set out the qualifications for the position in these terms: QUALIFICATIONS, SKILLS, ABILITIES AND EXPERIENCE Required 1. A degree in education or related discipline and/or journey interprovincial certificate or an equivalent combination of education and experience. 2. Two years recent/relevant work experience in designing, organizing and delivering/coordinating learning activities for adults. 3.

Understanding of and ability to apply the basic principles and practices of adult education. 4. Effective interpersonal, communication (oral and written), organizational and planning skills. 5. Ability to prepare, promote and negotiate training proposals. 6. Ability to counsel and advise on appropriate training design and instructional technique.

7. A valid Class 5 operator’s license. 8. Ability to work with people from equity groups. Desired Desired 1. Knowledge of business training resources and activities in Saskatchewan. 2. Ability to work independently and/or as a team member and to show initiative. 3. Ability to utilize computer systems to met ( sic ) word processing and spreadsheet analysis needs. 4. Working knowledge of SIAST Extension Policies and Procedures. [ 7 ] The “required factors” for a position are defined in

Article 1.44 of the Collective Agreement as “… the qualifications, skills, abilities and experience an employee must have to be successful in carrying out the responsibilities of the job. “Desired factors” are defined in

Article 1.16 as “... the qualifications, skills, abilities and experience that would be nice to have but would not be an essential requirement for carrying out the responsibilities of the job.” [ 8 ] The two grievors, Jim Marcia and Sharon Hopkins, both applied for the Continuing Educational Consultant position. Each submitted a letter and a résumé to SIAST. Ms. Hopkins was interviewed by a three-person panel on June 18, 2008. Mr. Marcia was interviewed by the same panel on June 19, 2008. III.

The Collective Agreement [ 9 ] The Collective Agreement contemplates that a candidate for a job opening must satisfy minimum requirements in order to be considered for appointment. If two or more candidates meet this threshold, SIAST may appoint the person with the strongest qualifications, skills, abilities and experience unless the two top candidates are relatively equal. In that case, the candidate with the most seniority must be appointed. [ 10 ] The relevant provisions of the Collective Agreement are set out below:

ARTICLE 1

DEFINITIONS In this agreement, unless the context otherwise requires, the expression: … 1.16 “ Desired Factors ” are the qualifications, skills, abilities and experience that would be nice to have but would not be an essential requirement for carrying out the responsibilities of the job. … 1.44 “ Required Factors ” are the qualifications, skills, abilities and experience an employee must have to be successful in carrying out the responsibilities of the job. … 4.7 Appointments 4.7.1 Appointments, supervisory assignments, promotions, voluntary transfers and voluntary demotions, as a result of an employment competition, will be made on the basis of the following factors: (

a) qualifications, skills, abilities and experience of the individual as it relates to the specific job for which selection is being done, and (

b) seniority (from either collective agreement). (

i) seniority of applicants from the other SIAST bargaining unit will be recognized if there are no qualified applicants from employees in the bargaining unit. 4.8 Relative Equality

4.8.1 Where the qualifications, skills, abilities and experience of two (2) or more applicants are relatively equal, seniority shall be the deciding factor. Candidates will be considered relatively equal if their final scores are within ten percent (10%) of each other. 4.9 Evaluation of Qualifications, Skills, Abilities & Experience 4.9.1 In evaluating the qualifications, skills, abilities and experience the employer shall do so in a way that is bona fide, fair, reasonable, non-arbitrary and non-discriminatory.

The employer shall not act in a manner such as to circumvent the legitimate role of seniority when developing and applying the above criteria. IV. The Interview Process [ 11 ] The members of the interview panel were Dennis Johnson, the Dean of SIAST’s Industrial Training Division, Lori Fredette, a SIAST Human Resource Consultant, and John Erickson, a Continuing Educational Consultant from SIAST’s Regina campus. An SGEU representative acted as an observer. [ 12 ] Ms.

Fredette took the lead in preparing an interview document referred to as the “Interview Guide.” It listed preliminary instructions to be given to the candidates at the interview and then set out a series of questions for them to answer.

The questions were organized under headings corresponding to the eight “Required” types of qualifications, skills, abilities and experience described in the job posting. [ 13 ] The responses to the questions were to be rated by the members of the interview panel and, in this regard, the Guide set out a number of points under each question that a candidate might be expected to mention in the course of providing his or her answer.

The Guide also contained spaces for the panel members to make notes about the candidates’ answers and places for them to record their score for each answer. [ 14 ] The answers were to be rated on a 5-point scale described in the Guide as follows: 0 for an unacceptable response; 1 for a poor response; 2 for a fair response; 3 for an average response; 4 for a good response; and 5 for an excellent response.

In order to be considered for appointment, candidates were required to score at least 60 percent on the interview, i.e. be awarded 60 percent of the total available points. [ 15 ] The scoring of the grievors, by panel member, is set out below: Panel Member Grievor Jim Marcia Grievor Sharon Hopkins Dennis Johnson 23 pts or 35% 28 pts or 45% John Erickson 34 pts or 52% 31 pts or 48% Lori Fredette 30 pts or 46% 31 pts or 48% Panel Average 29 pts or 44.9% 30 pts or 46% [ 16 ] As neither Ms. Hopkins nor Mr. Marcia received an overall score of 60 percent, both were ruled out of the selection process.

No further inquiries were made about their suitability for the position. V. The Arbitration Decision [ 17 ] Mr. Marcia filed a grievance alleging as follows: Senior qualified was not appointed to the position. [ 18 ] Ms. Hopkins’ grievance was expressed in these terms: Grievor was not offered the posted position despite having had worked in a like position without concern demonstrates her ability to do the job. Employer failed to evaluate her qualifications, skills, abilities and experience in a bona fide, fair, reasonable, non-arbitrary and non-discriminatory manner.

[ 19 ] The Arbitrator wrote extensive reasons and found a significant number of defects in the interview and assessment process. Her conclusions in this regard can be summarized as follows: (

a) SIAST’s use of a 60 percent threshold and its focus on the “average” ability of the candidates, as opposed to their ability to do the Continuing Educational Consultant job, amounted to a failure to respect

Article 4.7.1 of the Collective Agreement; (

b) The Interview Guide included questions relating to “desired factors” and not just “required factors”; (

c) The interview panel failed to draw any specific conclusions about why the grievors were not eligible for the position; (

d) The interview panel failed to understand the role that seniority played in the selection process; (

e) The interview panel erred by failing to look beyond the interview process to assess the qualifications of the grievors; (

f) The interview panel improperly applied the Interview Guide by failing to use prompting to elicit answers from the grievors; and (

g) The interview panel was not sufficiently flexible when giving the grievors marks for their answers to questions. [ 20 ] The Arbitrator upheld the grievances because of these concerns and then went on to find that both grievors had the requisite qualifications, skills, abilities and experience required for the Continuing Educational Consultant position. She directed the interview panel to complete the selection process in light of this conclusion. VI.

The Queen’s Bench Decision [ 21 ] As noted, SIAST brought an application for judicial review and asked the Court of Queen’s Bench to quash the Arbitrator’s decision. [ 22 ] The Chambers judge set aside her award. His key conclusions were as follows: (

a) The Arbitrator misconceived the importance and relevance of seniority in the selection process; (

b) The Arbitrator did not appreciate that SIAST’s use of the Interview

Guide reflected a 10-year history of such use and was mandated by the terms of the Collective Agreement. SIAST could not supplement the Guide with the more “comprehensive” approach to evaluation favoured by the Arbitrator; and (

c) The Arbitrator overlooked or misunderstood evidence as to why the interview panel had not used prompting questions during the interviews. In fact, it had taken that approach because of SGEU’s objection to the use of such questions. [ 23 ] The Chambers judge also dealt with the Arbitrator’s decision concerning the grievors’ qualifications for the Continuing Educational Consultant position. He concluded it was unreasonable because it lacked “justification, transparency and understandability.” VII. Analysis [ 24 ] The most efficient way to resolve this appeal is to examine the Arbitrator’s concerns one by one.

That is how I propose to proceed. [ 25 ] However, I should first note that there is no disagreement about the standard of review to be applied in resolving this appeal. The decision of the Chambers judge must be considered on the correctness standard. See: Saskatoon Credit Union Ltd. v. U.F.C.W., Local 1400 , 2009 SKCA 87 , [2010] 2 W.W.R. 105 at para. 27 . SIAST and SGEU agree that the Arbitrator’s decision itself is subject to review on the reasonableness standard. See: Saskatoon (City) v.

C.U.P.E., Local 59 , 2008 SKCA 11 , [2008] 6 W.W.R. 447. [ 26 ] This case also engages, to at least some extent, an issue concerning the amount of deference the Arbitrator should have shown in relation to SIAST’s assessment of the skills and qualifications of the job candidates. That issue surfaces only in the context of the final concern the Arbitrator raised about the work of the interview panel – her concern that the panel was not sufficiently flexible in scoring the grievors’ answers to questions.

As a result, I will leave the discussion of this point until I deal with the “Insufficient Flexibility in Scoring Answers” issue in the analysis that follows. [ 27 ] I turn, therefore, to an examination of each of the various errors the Arbitrator identified in the interview process. A. Use of “60 Percent” Threshold and Focus on “Average” Ability [ 28 ] The Arbitrator concluded that the requirement of a “60 percent” score on the interview was not a valid means of determining whether an employee satisfied threshold requirements with respect to qualifications, skills, abilities and experience.

The relevant part of her decision is set out below: [103] Counsel for the Employer submitted the 60% pass mark did not offend the sufficient ability or threshold test because of the marking system employed. The evidence established the answers to the questions were marked from 0 to 5: 0 was unacceptable; 1 was poor; 2 was fair; 3 was average; 4 was good; and 5 was excellent. Because the interview was based upon a total of 65 points, a candidate need only attain an average rating of 3 for each question to score 60%.

Then because the point guide provided that 3 was an average response, that was consistent with a sufficient ability or threshold standard. While there is some logic to the position advanced on behalf of the Employer, I am of the view that for the purposes of this collective agreement, relying on such logic is problematic. Average ability to perform the work in question is not the same as sufficient ability to perform the work in question. Similarly, assessing a candidate as average is not the same as assessing a candidate as able.

As noted earlier, the initial assessment required in relation to each candidate is whether that person can perform the work. Such assessment ultimately requires a response of “yes” or “no”. It is easy to understand what ratings of “no”, “good” or “excellent” mean. But what do “fair” and “average” mean in that context.

I find the utilization of a 60% passing mark in this case, together with the marking key followed, demonstrated a failure to appreciate and apply the threshold measurement standard, being the test prescribed by the collective agreement. [emphasis added] [ 29 ] The Chambers judge did not come to grips with this issue in the part of his decision dealing with whether the Arbitration had acted unreasonably in allowing the grievances. He did comment on it when dealing with the Arbitrator’s decision with respect to remedies. In that context, he said this:

[40] In paras. 102 and 103 of the Award, the arbitrator took issue with the applicant requiring a 60% score on the interview in order to be successful because she considered it offended the “sufficient ability” only requirement she found to exist in

Article 4.7. At para. 103, she noted there was some logic to the employer’s position of scoring the interview and requiring an average score to be successful, but then went on to state “…I am of the view that for the purposes of this collective agreement, relying on such logic is problematic. Average ability to perform the work in question is not the same as sufficient ability to perform the work in question….”

Article 4.7 does not refer to “sufficient ability”, but only ability, along with qualifications, skills and experience as they relate to the specific job for which selection is being done. [41] The problem with the arbitrator’s emphasizing that average ability is not the same as sufficient ability implies that ability which is sub-average is nevertheless sufficient ability for the position and, if this is so, raises the question how far sub-average can one be and still supposedly have sufficient ability to perform the job.

The arbitrator drew the distinction, but thereafter did not answer the question. [ 30 ] In my view, the Arbitrator’s conclusion in relation to the 60 percent threshold cannot be dismissed in this way. Her point was not that a “sub-average” candidate should be eligible for appointment. Rather, it was that the 60 percent rule, at least as applied here, did not determine whether a candidate met the requirements of the position. The Arbitrator was quite correct in this regard.

Let me explain why. [ 31 ] As indicated above, the assessment process proceeded on the basis that a candidate for the Continuing Educational Consultant position would be considered for appointment only if he or she scored at least 60 percent on the interview. This reflected SIAST’s “Staffing Procedure” document which provides as follows at pp. 9 and 10: Final scores will be expressed in percent terms. Final scores of each applicant will be compared. The concept of relative equality will be applied which means if applicants are relatively equal, then the senior applicant will be offered an appointment.

The definition of relative equality is that candidates will be considered relatively equal if their final scores are within 10% of each other. Applicants must score at least 60%, in the interview, to be considered. … [ 32 ] The details of how the percentage score for the interview was to be calculated are significant.

As noted earlier, the rating scale drawn from the Staffing Procedure document and applied to the answers to individual questions was described in the Interview Guide as follows: 0 Unacceptable (The Staffing Procedure document uses the phrase “Does not meet the factor.”) 1 Poor 2 Fair 3 Average (The Staffing Procedure documents says “Average, provides a sufficient number of expected responses or other responses.”) 4 Good 5 Excellent [ 33 ] Using this scale, the Interview Guide was structured with a view to allowing assessment of what it referred to as eight “Critical Factors.” Those factors tracked the eight “Required” factors listed in the job posting: 1.

A degree in education or related discipline and/or journey interprovincial certificate or an equivalent combination of education and experience. 2. Two years recent/relevant work experience in designing, organizing and delivering/coordinating learning activities for adults. 3. Understanding of and ability to apply the basic principles and practices of adult education. 4. Effective interpersonal, communication (oral and written), organizational and planning skills. 5. Ability to prepare, promote and negotiate training proposals. 6.

Ability to counsel and advise on appropriate training design and instructional technique. 7. A valid Class 5 operator’s license. 8. Ability to work with people from equity groups. [ 34 ] The first two factors, dealing with work experience and educational attainment, were listed in the Interview Guide but were not to be

scored on the 0 to 5 scale by the panel as only candidates with the requisite qualifications in this regard were selected for interviews. Somewhat curiously, however, the Guide expected the panel to grade, on a scale of 0 to 5, the candidates’ answers to the question “Do you hold a current, valid Class 5 operator’s license?” In any event, in relation to each of the six factors referred to in numbers 3 to 8 above, the Guide set out questions (four, three, two or one) designed to reveal whether the candidate satisfied the “Critical Factor” in question.

There were 13 questions in all, meaning that a candidate could earn as many as 65 points in total and, of course, meaning as well that the 60 percent threshold was set at 39 points. [ 35 ] The unusual feature of this approach is self-evident. A candidate could score a “5” on eight of the questions and a “0” (meaning “unacceptable” according to the Interview Guide and “Does not meet the factor” according to the Staffing Procedure document) on the other five and still be deemed to have met the threshold requirements for the Continuing Educational Consultant position.

Put in more concrete terms, and by way of example, imagine a candidate who scored 0 in relation to all of the questions relating to the “Critical Factors” described as (a) “ability to work with people from equity groups,” (b) “a valid Class 5 operator’s license,” and (

c) either “ability to counsel and advise on appropriate training design and instructional technique” or “understanding of and ability to apply the basic principles and practices of adult education.” If that candidate scored a 5 on the other questions, he or she would still have obtained more than 60 percent overall on the interview and, therefore, would have been found to meet the threshold requirements of the position. [ 36 ] Accordingly, it was clearly problematic for SIAST to use an “average” mark of 60 percent to determine whether the necessary qualifications for the Continuing Educational Consultant position had been met.

Simply put, that approach submerged or overlooked one of the central purposes of the interviews, i.e. the assessment of whether a candidate met the minimum requirements for appointment. As a result, there was nothing unreasonable or incorrect about the Arbitrator’s conclusion on this point. The scheme of the Interview Guide cannot be reconciled with the terms of the Collective Agreement because, as applied, it did not provide a means for determining whether a candidate satisfied the threshold qualifications, skills, abilities and experience required for the Continuing Educational Consultant position.

This error tainted the entire interview process. [ 37 ] This, of course, does not necessarily mean that the idea of a 60 percent threshold is invalid. If an interview guide is structured so as to make a candidate’s eligibility for appointment dependent on the candidate meeting each of the “required factors” relating to a position, it might be quite acceptable to require that the candidate also score an overall 60 percent rating as a prerequisite to appointment. However, that issue need not be decided here. [ 38 ] Thus, by way of

summary on this point, I conclude that the Chambers judge did not properly assess the “60 percent” issue. The Arbitrator made no reviewable error in this regard. B. Considering “Desired” Factors rather than only “Required” Factors [ 39 ] The Arbitrator also concluded that the Interview Guide was defective because, although it purported to address only so-called “Critical Factors,” it also effectively asked questions and/or expected answers dealing with “Desired Factors” referred to in the posting. As noted, those factors were listed in the job posting as follows: 1.

Knowledge of business training resources and activities in Saskatchewan. 2. Ability to work independently and/or as a team member and to show initiative. 3. Ability to utilize computer systems to met ( sic ) word processing and spreadsheet analysis needs. 4. Working knowledge of SIAST Extension Policies and Procedures. [ 40 ] The Arbitrator expressed her concerns on this point in the following way: [104] The next concern in relation to the use of the 60% passing mark is that a number of questions used in the interview guide actually addressed desired factors or included desired factors in the expected responses.

Question 5.2, “How do current issues regarding the future of education and training services in Saskatchewan impact the Continuing Education Consultant role?”, appears to be more in keeping with desired factor number 1, “Knowledge of business training resources and activities in Saskatchewan” than any required factor.

The expected responses for question 3.1, “If you received a training request from an employer, what parameters would you base your evaluation on?”, included numerous points of specific Employer protocols, which fall under desired factor number 4, “Working knowledge of SIAST Extension Policies and Procedures”. The expected responses for question 5.1, “Tell us about a time when you had to negotiate a contract.

What factors did you consider?”, included numerous points regarding specific Employer protocols, again falling under desired factor 4. [105] At least one of the questions, Question 3.1, “If you received a training request from an employer, what parameters would you base your evaluation on?”, included under critical factor number three, “Understanding of and ability to apply the basic principles and practices of adult education”, appears to have little relationship thereto and relates more to marketing and desired factor #4, “Working knowledge of SIAST Extension Policies and Procedures”.

[ 41 ] The Chambers judge made passing reference to these issues at para. 27 of his reasons for decision but did not deal with them further. [ 42 ] In my view, the Arbitrator’s reading of the interview questions, and her characterization of them, is reasonable. That said, there can be nothing wrong, in and of itself, with an interview panel asking a candidate questions relating to “desired factors.” After all, except for the situation contemplated by

Article 4.8.1 (relatively equal scores), SIAST is entitled to appoint the best candidate for a position. Questions designed to identify that candidate cannot be objectionable. [ 43 ] Accordingly, the difficulty with the Interview Guide in this case was not that it included questions and answers relating to the “Desired Factors” listed in the Guide.

The problem was that such matters were rolled in with those relating to “Required Factors” (or “Critical Factors” to use the language of the Interview Guide) and thereby obscured the question of whether a candidate met the threshold requirements for the Continuing Educational Consultant position. C. Failing to Draw Specific Conclusions [ 44 ] The Arbitrator appeared to believe the interview panel committed a reviewable error in its work because it did not draw “specific conclusions” about why the grievors were not qualified for the Continuing Educational Consultant position.

She said this at para. 106 of her decision: [106] Following the assessment process, while the panel made an overall determination of the mark to be assigned each candidate, the panel failed to draw any specific conclusions of why the candidates were not eligible for the position. [ 45 ] The reasons of the Chambers judge do not refer to this aspect of the Arbitrator’s conclusions. [ 46 ] In my opinion, the Arbitrator’s complaint that the panel did not draw “specific conclusions” about the grievors was an unreasonable criticism of its work. The panel had the Interview Guide and it used and applied the Guide.

Given that both grievors failed to obtain the threshold score of 60 percent, there was nothing more for the panel to do. The grievors’ scores said all that needed to be said. There was no obligation on the panel to go further and express “specific conclusions” about why the grievors were not qualified for appointment. D. Misunderstanding the Role of Seniority [ 47 ] The Arbitrator concluded that the interview panel’s work was open to review because it had not appreciated the role of seniority in the hiring process.

She wrote as follows, at para. 106: … Furthermore the panel failed to appreciate the right seniority afforded these internal candidates. That right is one of entitlement to the position if they have the qualifications, skills, ability and experience. Instead the panel disregarded this right, or alternatively, the panel members interpreted the right as one of entitlement if they passed the interview guide.

While passing an interview test may work effectively in some cases, it did not work here. [ 48 ] In order to put this aspect of the Arbitrator’s concerns in context, it is necessary to revisit the terms of the Collective Agreement. For ease of reference, Articles 4.7.1 and 4.8.1 are again reproduced here: 4.7.1 Appointments, supervisory assignments, promotions, voluntary transfers and voluntary demotions, as a result of an employment competition, will be made on the basis of the following factors: (

a) qualifications, skills, abilities and experience of the individual as it relates to the specific job for which selection is being done, and (

b) seniority (from either collective agreement). (

i) seniority of applicants from the other SIAST bargaining unit will be recognized if there are no qualified applicants from employees in the bargaining unit. 4.8.1 Where the qualifications, skills, abilities and experience of two (2) or more applicants are relatively equal, seniority shall be the deciding factor. Candidates will be considered relatively equal if their final scores are within ten percent (10%) of each other.

[49] Early in her reasons for decision, the Arbitrator correctly assessed the import of these provisions as being that: (

a) no candidate canbe appointed to a position unless he or she satisfies threshold requirements with respect to qualifications, skills, abilities and experience,and (

b) if two or more candidates meet the threshold, the best candidate may be appointed unless, as per

Article 4.8.1, theirqualifications, skills, abilities and experience are relatively equal. In that case, the candidate with the most seniority will be appointed. [50] Given the terms of the Collective Agreement, it is apparent that seniority had no role to play in the work done by the interviewpanel in this case. Neither grievor met the minimum qualifying standard and, as a result, seniority simply never became an issue.

TheChambers judge was obviously correct in concluding that the Arbitrator’s emphasis on seniority was “…misplaced and unreasonable inthe context of whether an internal candidate has the required skills, abilities, etc. for the specific job.” E.

Relying Exclusively on the Interview Process [51] The Arbitrator appeared to take the view that the shortcomings she perceived in an “interview only” evaluation process could havebeen overcome if the panel had gone beyond the interviews to check internal work records, performance reviews and so forth in order toobtain additional information about the qualifications of the grievors. In the end, the panel’s failure to take such additional steps wasseen by the Arbitrator as one of the reasons for allowing the grievances.

This is the key part of what the Arbitrator said in this regard: [107] A questionable assessment based upon one tool or procedure – such as an interview, may become reasonable if the hiring paneltakes additional steps to verify their conclusions. Often apparent gaps in ability can be filled in by cross references to other sources ofinformation such as résumés, references, supervisors, and internal work records and performance reviews. Alternatively, such sources if(sic) information may confirm the initial evaluation. No cross references or follow-ups were engaged in by the hiring panel.

Given theconsiderable degree of seniority of a candidate such as the Grievor Marcia, sole reliance on a single assessment tool, is not in my view areasonable or sound approach in light of his extensive work as an instructor – the primary drawing pool for candidates for the position inquestion.

With respect to the Grievor Hopkins, it would appear from her evidence, that the Employer, through its institutionalknowledge – that is the knowledge of her immediate supervisor, program head, continuing educational consultant, and or dean – wouldhave had information regarding her abilities that contradicted the hiring panel’s conclusions.

Again, this demonstrates the potential forerror when determinations with respect to specific rights are made based upon a single assessment tool. … [117] Relying upon a single assessment tool is not a sound practice when internal candidates have rights to positions by reason ofseniority or otherwise. … [52] In my view, the Chambers judge was quite right in finding that the Arbitrator’s conclusions on this point were unreasonable.

As hepointed out, SIAST and SGEU have had at least a 10 year history of using an interview guide to choose between candidates foremployment opportunities and SGEU made no objection to that technique being used in this case. The interview guide approach is alsodescribed in SIAST’s Staffing Procedure document. Moreover, and most significantly,

Article 4.8.1 of the Collective Agreement clearlycontemplates the use of such an evaluation technique by providing that seniority is the deciding factor in an appointment whencandidates are relatively equal and by then describing relative equality as being the situation where “… final scores are within tenpercent (10%) of each other.” This language makes sense only in the context of the sort of assessment technique employed in this case. [53] The interview system used by SIAST and reflected in the terms of the Collective Agreement is, of course, not unique.

The merits ofsuch an approach were explained by Arbitrator Rose in Temiskaming Hospital v. Ontario Nurses’ Assn. (Regan Grievance) (2006), (ON LA), 150 L.A.C. (4th) 334 (QL) at para. 27: … It is widely recognized that interviews are the most widely used tool in the selection process. Structured interviews are to bepreferred since they provide reliability, validity and are legally defensible.

The characteristics of structured interviews have beendescribed in the following terms (see Hackett, Rose and Pyper, “The Employment Interview: An Analysis of Canadian LabourArbitration Cases,” Labour Arbitration Yearbook 1999-2000, Kevin Whitaker, et al (eds.), Toronto: Lancaster House, 2000, at page 234). Several steps are required to structure the interview. First, a comprehensive analysis of the job is conducted. The criteria relevant to theperformance of the job are then determined from the job analysis and then weighed according to their relative importance.

Interviewquestions are derived from these criteria, and a formal scoring guideline is developed describing possible responses and correspondingscores. The response to each question is scored individually and the total for the interview is summed at the end. A panel of trainedinterviewers is used and the interview is standardized, i.e., the same person asks all applicants the same questions in the same order. Structured interviews enhance interview reliability and validity. Interview reliability refers to the consistency of scores given to jobapplicants by interviewers.

As noted by Hackett, et al, at page 235: Where two or more interviewers are assessing the same candidate, either sequentially (i.e., separate interviews) or simultaneously (i.e.,panel interview), we would expect similarity in ratings of the same attribute(

s) for the same candidate. Where there is little convergencein interviewer ratings, this suggests that the ratings reflect the interviewer attributes rather than candidate attributes. Technically, thisdivergence in interviewer ratings represents measurement error. Accordingly, one means of assessing the integrity of an employmentinterview with two or more interviewers is to assess the consistency (reliability) of ratings across interviewers and across interviewees.

Interview validity refers to how well the interview score predicts future job performance. [ 54 ] Accordingly, it was unreasonable for the Arbitrator to overlook the Collective Agreement and impose her own sense of the most effective assessment system on the established relationship between SIAST and SGEU. The selection process in this case was not invalid because SIAST did not check with references and supervisors or consider internal work records and performance reviews before deciding whether the grievors met minimum job requirements. F.

Failing to Use Prompting Questions [ 55 ] The Arbitrator concluded that “the most blatant flaw in the hiring process was the rigid use of the [interviewing] tool itself.” Her first concern on this point was that the panel had not used prompting questions. She expressed her concerns as follows: [109] Addressing the rigid use of the “Interview guide” as the assessment tool, I have already commented above on some perceived flaws. However, a significant failure in the use of the tool itself was the panel’s failure to utilize prompting questions during the interview.

Prompting can be measured and applied in a consistent fashion. Prompting some candidates and not prompting other candidates does not necessarily result in treating the candidates unfairly. Fair and consistent treatment of candidates does not necessarily require treating candidates exactly the same. Candidates often come to the assessment process with different or varied qualifications, skills, abilities and experience that must be taken into account throughout the assessment process. The object of any assessment is to arrive at an accurate, or true, measure of each candidate’s ability to perform the work.

Assessing candidates is not a game to be measured only by how skilful a candidate is in scoring well on a series of tests. [ 56 ] In the circumstances here, this criticism of the panel is misplaced. The Arbitrator’s comments with respect to the use of prompting questions is grounded in her account of the evidence of Heather Sully, a senior recruitment and retention specialist with SIAST. At para. 44 of her decision, the Arbitrator summarized Ms. Sully’s evidence in this way: [44] Ms. Sully also appeared to be an advocate of the use of probing questions in the interview process.

She stated that utilizing structured probing, that is open-ended, would make an interview process more valid because more information would be collected. It also allowed for more rapport between the candidate and the interview panel. However, she ruled out the use of probing questions for in-scope employees because it would result in treating those employees differently. Another distinction Ms.

Sully made between interviews of in-scope and out-of-scope employees was that only out-of-scope employees would get credit if after failing to score well on one question, they subsequently demonstrated they had that specific knowledge or experience in responding to another question. [emphasis added] [ 57 ] However, when the judicial review application was argued in the Court of Queen’s Bench, Ms. Sully filed an affidavit saying the Arbitrator’s account of her evidence was not accurate.

She indicated that she had not said she had ruled out prompting questions but, rather, had said such questions were prohibited because of past objections by SGEU to their use. She said SGEU had opposed these kinds of questions because it believed they resulted in differential treatment of candidates and, therefore, unfair treatment of them. Lori Fredette filed an affidavit to the same effect. SGEU accepted Ms. Sully’s and Ms.

Fredette’s explanation of why prompting questions were not used in the interview process. [ 58 ] As a result, it is apparent that this aspect of the Arbitrator’s decision was based on a misapprehension of the evidence and, thus, was unreasonable. G. Insufficient Flexibility in Scoring Answers [ 59 ] The Arbitrator’s second criticism of the panel’s use of the Interview Guide was that panel members were not sufficiently generous in giving credit for answers provided by the grievors.

She explained her thinking as follows: [110] In this case, in addition to failing to use prompting, the members of the hiring panel were not sufficiently flexible in giving credit to the Grievors for responses they gave.

As noted from the recitation of the evidence of the individual members of the hiring panel, they failed to give marks, or gave low marks, where they did not care for the example given; they failed to give marks where the Grievors approached a question from a different perspective; they failed to give the Grievors credit for a baseline of knowledge when that would appear to be a given based on their responses and experience; they drew conclusions that were unwarranted in light of the questions and responses; and in the case of Mr.

Marcia they drew conclusions regarding his character and attitude from limited information and did not verify their conclusions. Also in Mr. Marcia’s case, they deducted points from an otherwise suitable answer because he allegedly disclosed the name of an employee contrary to notions of confidentiality that the hiring panel held. In this regard, there was no evidence that the identity of the individual was readily ascertainable; it was certainly not revealed.

Given the nature of the question itself and the confidentiality of the hiring process, I am unable to conclude the panel’s concern was a valid one. [ 60 ] The decision of the Chambers judge does not speak directly to these issues.

[61] As indicated earlier, the Arbitrator’s approach raises a question about the amount of deference, if any, she was obliged to showSIAST in relation to its assessment of the grievors’ qualifications, skills, abilities and experience with respect to the ContinuingEducational Consultant position. (I note that this is a separate question from the one about whether the stated job requirements arethemselves proper.) [62] There is presently a division of opinion in the arbitral jurisprudence on the question of the standard of review an arbitrator shouldapply when considering an employer’s assessment of whether a candidate satisfies the established requirements for a position.

Somearbitrators favour a deferential approach and some do not. The situation is summarized as follows in Brown and Beatty, CanadianLabour Arbitration, loose leaf, 4th ed., vol. 1 (Aurora: Canada Law Book, 2009) at 6:3100: … with respect to the application of whatever standards could reasonably be expected of the competing applicants, arbitratorstraditionally rejected the notion that the arbitral function is in the nature of an appeal or fresh determination on the merits.

Rather, theyusually deferred to management’s application of the relevant criteria for a particular job where it was established they had applied thosestandards fairly, honestly and without malice or ill will toward any of the candidates. However, following a decision of the Ontario Divisional Court [Canadian Food & Allied Workers Union, Local 175 v. Great Atlantic& Pacific Co. of Canada Ltd. (1976), 76 C.L.L.C. 14,056 (Ont. Div. Ct.); leave to appeal refused (1976), (ONSCDC), 13 L.A.C. (2d) 211n. (Ont. C.A.)], many arbitrators have questioned this approach.

In their view, this limited standard of reviewshould not prevail, on the second component of the employer’s decision, if the parties have not, either in the seniority provisions or in themanagement’s rights clause of the agreement, specifically provided that the determination of an employee’s abilities or qualifications isto be based on “the opinion or judgment” of the employer.

Although in many agreements such a phrase is included to preservemanagement’s discretion, in agreements where it is not, these arbitrators have insisted that the scope of arbitral review with respect to theemployer’s application of the relevant standards against which an employee’s ability is measured ought to be more exacting.

Their rule,which has not been embraced by all arbitrators, is that an arbitrator may fully review an employer’s decision on its merits to determinewhether, on its own standards and procedures, the employer made the correct decision as to the ability and qualifications of thecompeting applicants. [footnotes omitted] [63] In this case, the Arbitrator concluded, at the outset of the “Decision” part of her reasons, that she should apply “the broad standardof correctness” standard of review with respect to the issue of SIAST’s assessment of the grievors’ qualifications, skills, abilities andexperience.

This said, and relying on the arbitral decisions referred to in the next paragraph, she also acknowledged that an employer isoften in a better position than an arbitrator to evaluate the qualifications, skills, abilities and experience of its employees.

Accordingly,she concluded she should “tend to defer to [SIAST’s] judgment and should have a firm basis for concluding [SIAST’s] evaluation wasnot correct” before intervening. [64] I must say that, for my part, I find it difficult to understand the authorities relied on by the Arbitrator and, specifically, the decisionsof Arbitrator Pelton in Communications, Energy, and Paperworkers Union of Canada, Local 649 v. Saskpower, (SKL.A.) and Saskatchewan Union of Nurses v. Regina Qu’Appelle Health Region (Humphreys Grievance) (2004), (SKLA), 139 L.A.C. (4th) 244 (QL).

They suggest that an arbitrator should employ a “broad standard of correctness” standard of reviewwhile, at one and the same time, also showing considerable deference to the decisions of the employer. These are contradictorystandards. Correctness involves no deference whatsoever on the part of a reviewing authority. Deference, of course, involves deference. [65] In any event, the basic error made by the Arbitrator on this issue is not difficult to identify.

The Collective Agreement itself speaksto the applicable standard of review to be applied by an arbitrator in circumstances relating to the employer’s assessment of thequalifications, skills, abilities and experience of candidates for job postings. As will be recalled,

Article 4.9.1 of the Agreement states asfollows: 4.9.1 In evaluating the qualifications, skills, abilities and experience the employer shall do so in a way that is bona fide, fair, reasonable,non-arbitrary and non-discriminatory. The employer shall not act in a manner such as to circumvent the legitimate role of seniority whendeveloping and applying the above criteria. [66] In my view, this provision unavoidably implies that SIAST’s evaluation of a job applicant’s qualifications, skills, abilities andexperience is not to be reviewed by an arbitrator on the standard of correctness.

As Arbitrator Priel said in Saskatchewan Institute ofApplied Science and Technology v. The Saskatchewan Government and General Employees Union (Yvonne Ross Grievance) (April 11,2002), unreported, at p. 9,

Article 4.9 serves as “… a yardstick for use by an arbitrator in reviewing an employer’s decision with respectto qualifications in posting a grievance.” Accordingly, SIAST’s assessment of the qualifications, skills, abilities and experience of acandidate should not be overridden by an arbitrator unless that assessment is something other than bona fide, fair, reasonable, non-arbitrary and non-discriminatory.

[ 67 ] The Arbitrator in this case was bound to work within the framework of the Collective Agreement.

As a result, it is clear that she erred and acted unreasonably by overlooking the terms of the Agreement and proceeding, instead, to rely on authorities which speak only generally to the standard of review or speak to it in the context of differently worded collective agreements. [ 68 ] There are aspects of the Arbitrator’s assessment of the interview panel’s work which suggest she was prepared to review the interviews of each grievor on something close to a question-by-question, panel member-by-panel member basis and to substitute her own assessments of the quality of the grievors’ answers for the assessments of the panel members.

Some indication of her thinking in this regard is apparent from her comments to the effect that the panel “failed to give marks where the Grievors approached a question from a different perspective” and that it “failed to give marks, or gave low marks, when [it] did not care for the example given.” It is difficult to see any deference in that line of analysis. [ 69 ] In light of all of this, I conclude that no effect can be given to the Arbitrator’s concerns about how the interview panel assessed the qualifications, skills, abilities and experience of the grievors.

She acted unreasonably by overlooking the terms of the Collective Agreement dealing with the standard of review she was obliged to work with and it seems clear that, in at least some respects, she erroneously applied a standard in the nature of “correctness” when considering the work of the panel. VIII. Conclusion [ 70 ] The Chambers judge correctly determined that several features of the Arbitrator’s decision were unreasonable.

However, in my respectful view, he failed to deal adequately with the way in which the “60 percent” threshold requirement was incorporated into the Interview Guide and applied by the panel. There was an obvious flaw in this aspect of the interviews.

Thus, notwithstanding her reviewable errors in relation to other points, the Arbitrator did have a proper basis for reaching her bottom line conclusion that the interview process was flawed and inconsistent with the demands of the Collective Agreement. [ 71 ] That said, I agree with the Chambers judge that the Arbitrator acted unreasonably in deciding both grievors were qualified for the Continuing Educational Consultant position. As he points out at paras. 37 to 39 of his decision, the Arbitrator’s analysis was conclusary in nature.

In the part of her reasons where she recited the evidence, the Arbitrator did refer, at some length, to the qualifications and experience of the grievors. But, she ultimately failed to relate those matters to the “required” factors for the Continuing Educational Consultant position.

Further, and in any event, her conclusions on the matter also seem to have been underpinned by facts and considerations outside of what would have been canvassed in the interview process. [ 72 ] In the end, therefore, the appropriate remedy here is to allow SGEU’s appeal but only to the extent of confirming the Arbitrator’s decision to uphold the grievances on the basis that the interview process was fatally flawed. [ 73 ] SGEU is entitled to costs in the usual way. DATED at the City of Regina, in the Province of Saskatchewan, this 2 nd day of May, A.D. 2011. “RICHARDS J.A.” RICHARDS J.A.

I concur “LANE J.A.” LANE J.A. I concur “OTTENBREIT J.A.” OTTENBREIT J.A.