1. Reasons for recognizing competent Career Guidance practitioners
Career guidance practitioners play a fundamental role in fostering adults participation in education (Council of European Union 2004, 2008a; European Ministers of Vocational Education and Training, and the European Commission 2002; Lisbon European Council 2000) , but in most European countries the qualifications of careers guidance practitioners are mainly based on informal and not formal learning and not formally recognized (CEDEFOP 2009). This reduces trust in career guidance services and makes problematic hiring career guidance practitioners.
In the last years several frameworks for officially recognizing competent career guidance practitioners have been developed, amongst the others EVGP (IAEVG no date) , MEVOC (MEVOC website 2011), EAF (Evangelista 2011), NVQ 3 Advice and Guidance (ENTO 2006). We define here competent worker: a person able to carry out an occupation or job task up to a predetermined standard . Official recognition can take several forms described later.
This article highlights the issues involved in recognizing competent workers in general and from this we discuss the structure of the frameworks addressed to career guidance practitioners listed above.
This article is based on the Study on Existing Frameworks to Validate Competence of Career Guidance Practitioners written by the Author (Evangelista 2011) and on the IMPROVE Guidelines (IMPROVE Project partners 2011), both produced in the project IMPROVE Improving Validation of Not-Formal Learning in European Career Guidance Practitioners. IMPROVE is a European Project aiming to develop and test guidelines for recognition of competence of career guidance practitioners (IMPROVE website 2011).
In many occupations competence of workers is assured by requiring a specific educational and/or vocational qualification. This approach is increasingly recognized as not fully satisfying. It proves inadequate in the many sectors where formal paths of education don’t exist or are not widespread, as in the case of career guidance. Moreover request of a formal qualification is discriminating with whom has not time or money to attend a formal educational course, has a preference for learning outside a classroom or has gained an educational qualification (for example in another country) that is not recognized. Finally, the technical knowledge and transversal cognitive skills related with formal qualifications are only part of the factors by which good performance depends.
In the last years the European Commission has started asking member states to develop systems for recognition of key skills and competencies, including those acquired mainly outside the formal learning system ( Council of the European Union 2008b) , and several new frameworks have been developed by public authorities and private organizations. Some of them are general, i.e. apply to a large number of occupations, some others are focused only on career guidance.
2. Structure of frameworks
EVGP Educational and Vocational Guidance Practitioner is the framework for accrediting career guidance practitioners at international level launched in 2007 by IAEVG International Association for Educational and Vocational Guidance. The framework accredits the Educational and Vocational Guidance Practitioner (IAEVG no date). The framework is based on 91 elements, of which 11 are compulsory and other 7-10 can be chosen by the candidate. Some minimum requirements about educational qualification and experience are compulsory. Assessment is carried out based on examination of candidate’s narrative of professional biography, control of possession of educational qualifications and other educational or training certificates, control of possession of proven experience, self assessment of the elements.
MEVOC is a framework created through a European project with the same name coordinated in 2003-2006 by IBW Institute for Research on Qualification and Training of the Austrian Economy (MEVOC website 2011). The framework allows to get a European Certificate For Career Guidance Counselors (Petermandl, no date). The framework is based on 35 elements (140 in the full version). A minimum requirement of experience is compulsory. Assessment is carried out based on online (web based) test (focused on the specialist and methodical knowledge relevant for career guidance counselors), Assessment centre tests (focused on transversal skills, see a definition below), written paper (focused on theory of educational counselling and career guidance) (Petermandl, no date).
EAF European Accreditation Framework For Career Guidance Practitioners is a framework developed by the Author, following his participation in the European Project EAS European Accreditation Scheme for Careers Guidance Counsellors carried out in 2006-2008 (Evangelista 2008). The framework allows to get the EAF Certificate on one or more than one of the 3 main tasks (elements) of the framework: 1. Deliver information related to career guidance as a separate activity , 2. Perform career guidance interviews , 3. Carry out career guidance activities in small groups. For assessment purposes, each of the main tasks is divided in several sub elements (subtasks) with a tree roots structure. In the current version assessment is carried out with examination of candidate’s CV and the PFI Performance Focused Interview (see a definition below). The framework is going to be used in the IMPROVE project.
NVQ 3 Advice and Guidance has been developed by Employment NTO, an English organization in charge of developing and maintaining the UK National Occupation Standards for Career Guidance, and made operative in 2002. An update has been produced in 2006 and a new version is currently being developed. The framework allows to get an award in Advice and Guidance at several levels of expertise, here we’ll refer to level 3 and to the 2006 version. The 2006 version is based on 30 elements (tasks). For assessment purposes, each of them is divided in several sub elements (subtasks) with a tree roots structure. Assessment is carried out based on assessment methods agreed with candidate (Read 2006). Usual means are direct observation of the person whilst carrying out his/her work, professional discussion, testimonies from colleagues and supervisors, examination of documentation produced by the person whilst carrying out his/her work.
All existing frameworks for recognizing competent workers (from now onward we call them validation frameworks), addressed either to occupations in career guidance (as that briefly described at previous chapter) or to all occupations are made of four components:
- A. What is recognized or granted at the end of the assessment procedure
- B. The elements the candidate has to possess or master to be recognized competent
- C. The tools to collect the evidences that demonstrate possession or mastering of the elements
- D. The assessment procedure.
3. What is recognized or granted
Official recognition can take the form of a title, a qualification, an attribute to a qualification, or a certificate. For example IAEVG framework grants the title of Education and Vocational Guidance Practitioner, the Sistema Toscano delle Competenze (Tuscany Italian Region general validation framework) grants the qualification of Consulente di orientamento (career counselor) (Evangelista 2009), the Europsy framework addressed to psychologists allows to become Registered EuroPsy Psychologist (here we have the attribute Registered EuroPsyadded to a the psychologist qualification) (EFPA website 2011), while MEVOC grants the European Certificate for Career Guidance Counselors.
4. The elements the candidate has to possess or master
The elements the candidate has to possess or master to be recognized competent can be personal features such as specific knowledge, transversal skills, technical skills, attitudes, character traits, etc. Examples of personal features in EVGP are Social and cross-cultural sensitiveness and Ability to communicate effectively with colleague or clients, using the appropriate level of language , and in MEVOC Thinks positively (also in less encouraging situations) and To plan and manage activities .
The elements can also be job tasks to master, in this case also sub elements (sub tasks) are taken in consideration. In EAF one of the elements (main tasks) the practitioner has to master is 2. Perform career guidance interviews and some of the related sub elements are 2.1. Use appropriate interview techniques to manage the interview, 2.2. Assist clients to identify professional or learning goals, 2.3. Assist clients to determine action plans related to work or learning, 2.4. Assist clients to implement the action plans developed, including job search.
In NVQ 3 Advice and Guidance the element AG 2 Support clients to make use of the advice and guidance service has the sub elements 2.1 Enable clients to decide whether to use the service , 2.2 Agree with clients their use of the service ; 2.3 Identify and provide the information required by clients .
The elements are identified through a job analysis, i.e. a systematic study of the tasks that are performed in a work role. The actions are identified drawing a flowchart describing how a job is carried out and this way main tasks and tasks are described as a tree root, where combination of simpler actions allow to carry out the more complicate. The job analysis allows identifying a hierarchy of tasks, from the most important and general (main tasks) to the minor ones (tasks and sub tasks). Then personal features needed to carry out each task are also inferred.
There are basically two main approaches in the choice of the elements of a framework. In the first approach the elements considered worth and requested to a competent worker are the antecedents of good performance, that is to say the personal features causally related with good performance (these are specific knowledge, transversal skills, technical skills, attitudes, character traits). Here the assessment is based in checking possession of these elements. In the second approach the elements are specific job tasks that the practitioner has to master and the assessment is instead based on assessing or reconstructing performance. In this last case work performance is analyzed starting from the minor tasks.
An important point to stress is when dealing with assessing performance of people at work, the performance based approach scores best, because the other is indirect, that is to say speculative. If the candidate posses the prescribed personal qualities than probably will be capable to carry out the main task(s) for which accreditation is sought. The performance based approach instead focuses and assesses directly what we are interested in, good work performance that is to say competence at work.
This is probably the reason why no framework is based only on personal features. EVGP and MEVOC are mixed, some of their elements are personal features while other are job tasks. For example in EVGP together with personal features we find job tasks such as: Accurately and thoroughly conceptualize and diagnose clients’ needs based on different assessment tools and techniques, Identify situations requiring referral to specialized services , Maintain up-to-date listings of referral sources , etc. The same in MEVOC, where together with personal features we find job tasks such as Identify clients’ training needs to advance their career, Analyse clients’ needs, Interviewing , etc.
We think this mixture depends by lack of a rigorous theory when developing these frameworks. Both EVGP and MEVOC have been refined by asking experts or practitioners what they thought important to be a good career guidance practitioner, and these listed factors following different approaches.
EAF and NVQ 3 Advice and Guidance are both performance based because the elements the practitioner has to possess or master are all job tasks.
In performance based frameworks the elements the practitioner has to master are different from the one that are assessed. Mastering of main tasks is assessed examining performance in minor tasks and by assessing other factors, usually knowledge and attitudes.
Direct inquiry of knowledge (in itself an element of the personal features based approach) it is useful to avoid to wait for observation of the performance where that knowledge is used, so it can be used to shorten the assessment of performance.
For example in assessment of sub element 2.3 Identify and provide the information required by clients of NVQ 3 Advice and Guidance the practitioner must know the types of information clients seek and the different reasons they have for seeking it, why it is important to confirm the information required, etc.
The same in EAF, where in the assessment of element 2. Perform career guidance interviews knowledge by the practitioner of career guidance information is checked.
As for attitudes, it can be decided that performing up to the requested standard cannot be enough, but the practitioner must have specific attitudes embedded in his/her practice such as specific attention to privacy, gender issues, constant improvement of own knowledge and skills, etc. Again, it can take too long to spot these observing performance, so to save time it can be worth to focus assessment directly on these attitudes.
In EAF for example assessment is focused directly also on attention to privacy and ethical practice, constant improvement of own knowledge and skills, development of clients’ career skills.
5. The tools to collect the evidences
The evidences demonstrate possession or mastering of the elements identified in the previous chapter. The evidences can be already existing (such as educational certificates or documentation produced by the person whilst carrying out his/her work ) or produced on purpose for being assessed (for example testimonies from colleagues and supervisors).
For example in EVGP a degree in psychology is considered an evidence of possess of some specific knowledge useful for working as a Career Guidance practitioner. Evidences of mastering specific job tasks can result by direct observation (the evidences here are the actions the practitioner does), by discussion of case studies, by interviewing the candidate to validation on how he/she perform the job tasks (in these cases the evidences are the answers the practitioner gives), etc.
The evidences demonstrating possess of specific personal features can be collected/elicited by the following assessment methods:
- 1. Examination of candidate’s CV or of narrative of professional biography
- 2. Control of possession of educational qualifications and other educational or training certificates
- 3. Control of possession of proven experience
- 3. Interview or written test about technical knowledge
- 4. Tests (skills, personality, attitudes, etc.)
- 5. Role Playing focused on transversal skills (such as the Leaderless Discussion in the Assessment Center )
- 6. Interview focused on transversal skills (as in BEI Behavioral Event Interview).
The Leaderless Discussion is a group discussion for which a formal leader has not been designated and in which an instructor does not participate. Participants are observed by assessors and their behavior evaluated.
In the Behavioral Event Interview (BEI) the interviewee describes, in his/her own words, what he/she said, thought, felt, and did in six episodesthree positive and three negativeat work.
In both Leaderless Discussion and BEI personal characteristics are coded both for frequency of occurrence and for the level of complexity.
The evidences demonstrating mastering of job tasks can be collected/elicited by the following assessment methods:
- Direct observation of the person whilst carrying out his/her work
- Professional Discussion
- PFI Performance Focused Interview
- Discussion of case studies
- Testimonies from colleagues and supervisors
- Testimonies from clients
- Examination of documentation produced by the person whilst carrying out his/her work
- Examination of Portfolio of Work
- Simulation of job tasks.
A Professional Discussion can be defined as a n interview conducted between an assessor and candidate (assessed person), in which the candidate describes his/her job tasks and how his/her performance achieves requirements set by standards.
The PFI is a structured Professional Discussion in which all the candidates are asked the same list of questions and the questions are focused on predetermined aspects of performance.
A Portfolio of Work is a collection of products of a person’s work showing mastering of some job tasks.
Professional Discussion and PFI differs from BEI because in BEI the questions are focused on personal features and not on performance itself.
Usually the validation frameworks are based on evaluation of the practitioner by an evaluator. This is considered more reliable than self evaluation, but some frameworks include also some practitioner’s self evaluation. Usually the self evaluation has to be supported by other documents or testimonies.
For example in EVGP mastering of job tasks and possess of transversal skills is carried out by self evaluation, and for each of these the practitioner has to provide documentation supporting his/her conclusion. It can be documentation proving attendance to a formal training course, testimonies by people who have observed the practitioner or other suitable means.
Self assessment without documents or testimonies to support it is useful to check its own knowledge and learning process and it can be used as a preparation for the assessment, as the MEVOC Self Assessment Tool (MEVOC website 2011).
EAF and NVQ 3 Advice and Guidance make use only of external evaluation.
Every framework uses a specific set of tools to collect or elicit evidences (see chapter 2). EVGP and EAF use tools collecting information about personal features and proving mastering of job tasks, MEVOC only tools focused on personal features, NVQ 3 Advice and Guidance only tools focused on mastering of job tasks.
6. The assessment procedure
The assessment procedure sets how to structure the process, which evidences to collect, by which tools and how to score them, how to choose, train and supervise the evaluators and so on.
The main challenge in every assessment procedure is to find a workable compromise between efficacy and weight of the assessment procedure. A procedure may be very effective but if it requires significant dedication of time and economic resources it will have minimal possibility to become established and widely implemented (unless it is enforced by a public authority). On the other hand, a procedure which requires little time, but is less effective also presents the weakness of minimal utility (IMPROVE Project Partners 2011).
7. Sources of error in frameworks
Errors cause the procedure of assessment to be of little or no effectiveness and results not reliable. Errors can result from choices regarding components described at chapters 4.The elements the practitioner has to possess or master , 5.The tools to collect the evidences , 6.The assessment procedure.
Error 1. Poor individuation of the elements the practitioner has to possess or master. This occurs when the chosen elements A. are external to the sector, B. are not clear, too generic or too general, C. can be taken for granted, D. have a weak relation with performance or refers to minor tasks.
As for point C, there are dozens of factors that can harm job performance. For example poor ICT skills, poor writing and communication skills, poor personal hygiene, attention deficits, substance addiction, specific chronic diseases, inability to cope with stress and frustration, lack of punctuality, bad temper, restlessness, tardiness, inability to summarize, dishonesty, and many others. Luckily practitioners usually are at acceptable standard, so is not worth to focus assessment on them. C and D are the reasons why EVGP lists 91 elements and MEVOC full version 140.
We list some elements taken from the diverse frameworks as exemplas:
- 4.6.5 . Assist clients in sexual identity (EVGP). This is clearly outside the career guidance sector (error 1.A.).
- S pecific skills for handling difficult target groups (MEVOC). Which skills? Which target groups? The element appears too generic and general (1.B.)
- Basic telephone communication (MEVOC) (1.C.)
- Not being afraid of new experiences or changes (MEVOC), AG 17. Provide support for other practitioners (NVQ 3 Advice and Guidance) (1.D., weak relation with performance)
- 2.1 Enable clients to decide whether to use the service, 3.3 Bring interactions to an end (NVQ 3 Advice and Guidance) (1.D., minor tasks)
Points A, B direct assessment towards elements external to the sector or not clear or too much aggregate, so the results of the assessment are not significant or not reliable.
Points C, D bring a multiplication of elements. This too affects assessment, because the more elements you have to consider, the less accurate or the more cumbersome the process will be.
Error 2. Poor choice of the tools to collect the evidences. This error occurs when the tools to collect the evidences are not coherent with the elements. Perhaps the most evident error of this kind is in MEVOC: many elements of the framework are job tasks, but none of the three tools for assessment ( on online test, Assessment centre tests, written paper) check performance on these.
Error 3. Poor assessment procedure. This occurs when the tools to collect the evidences are right, but the assessment procedure has flaws. For example the tools to collect the evidences are not used properly, evidences are not examined well, the scoring system is not precise, etc.
It is possible to draw some principles to make the assessment procedure more effective and impartial:
- Assessment has to include a face to face examination of the practitioner
- Systematic inquiry (as opposed to sample inquiry) of elements is to prefer
- The assessment process, the assessment tools and the content of the tests and interviews (for example case studies or questions in interviews) must be the same for all the practitioners
- The scoring has to be based on explicit and measurable criteria
- The assessment procedure, the assessment tools and the scoring criteria must be applied uniformly by all the evaluators
- The assessment procedure and the assessment tools must be structured and managed in a way the practitioners must not have the possibility to cheat.
References
CEDEFOP. Professionalizing Career Guidance. Practitioners competences and qualification routes in Europe . Luxembourg: Office for Official Publications of the European Communities, 2009.
Council of the European Union. Resolution of the Council and of the representatives of the Member States meeting within the Council on Strengthening Policies, Systems and Practices in the field of Guidance throughout life in Europe, 2004. Retrieved on April 3, 2011 at http://ec.europa.eu/education/policies/2010/doc/resolution2004_en.pdf.
Council of the European Union. Resolution of the Council and of the Representatives of the Governments of the Member States, meeting within the Council of 21 November 2008 on better integrating lifelong guidance into lifelong learning strategies, 2008a. Retrieved on April 3, 2011 at http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:2008:319:0004:0007:EN:PDF.
Council of the European Union. Council conclusions of 22 May 2008 on adult learning, 2008b.( Measures to be adopted by the Member States, with the support of the Commission, Annex, point 8). Retrieved on August 1, 2011 at http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:2008:140:0010:0013:EN:PDF.
ECGC website retrieved on February 3, 2011 from http://www.ecgc.at/index.php?option=com_content&task=view&id=5&Itemid=13.
EFPA website. Retrieved on August 1, 2011 at http://www.efpa.eu.
ENTO Employment NTO. National Occupational Standards for Advice and Guidance, 2006.
European Ministers of Vocational Education and Training, and the European Commission. The Copenhagen Declaration, 2002. Retrieved on April 3, 2011 at http://ec.europa.eu/education/pdf/doc125_en.pdf.
Evangelista, L. How EAF Accreditation Framework for the European Career Guidance Practitioners was developed, 2008. Mimeo
Evangelista, L. La valutazione e la certificazione delle competenze nel Sistema Toscano delle Competenze, 2009.
Evangelista, L. Study on Existing Frameworks to Validate Competence of Career Guidance Practitioners. Version 2 dated 24 April 2011 , 2011. Mimeo. Retrieved on August 1, 2011 at http://www.improveguidance.eu/sites/default/files/Evangelista_2.pdf.
IAEVG. Application for Educational and Vocational Guidance Practitioner EVGP, no date. Retrieved on February 3, 2011 at http://www.cce-global.org/Downloads/EVGP/app-en.pdf.
IMPROVE Project partners. IMPROVE Guidelines for the validation of competence of people at work (version 11 July 2011) , 2011. Retrieved on August 1, 2011 at http://www.improveguidance.eu/sites/default/files/Guidelines.pdf.
IMPROVE website. Retrieved on August 1, 2011 at http://www.improveguidance.eu.
Lisbon European Council. Presidency Conclusions, 2000. Retrieved on April 3, 2011 at http://www.europarl.europa.eu/summits/lis1_en.htm.
MEVOC Standards. Retrieved on February 3, 2011 at http://www.mevoc.net/EN/htm/fs.htm.
MEVOC website. Retrieved on February 3, 2011 at http://www.mevoc.net/EN/htm/fs.htm.
Petermandl, M. ECGC Strategy Paper, no date. Retrieved on February 3, 2011 at http://www.ecgc.at/images/stories/SPAPER/ecgc-strategy%20paper-en.pdf.
Read, H. Excellence in assessing. Reads On Publications, 2006.
This document has been produced by © Leonardo Evangelista for the project IMPROVE Improving Validation of Not-Formal Learning in European Career Guidance Practitioners 510640-LLP-1-2010-1-IT-GRUNDTVIG-GMP (2011-2012) http://www.improveguidance.eu. This document reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. This article has been opublished on Revista de pedagogie no. 59 (3) 2011, ISSN 0034-8678.