Measurement & Assessment

YOWOMO2.0 developed two ideas for measurement and assessment of competences (download pdf version) .

For a more formative assessment (e.g. at the start of the training or for the monitoring of the learning process) there are quizzes you can find in the footer section. The quizzes will guide you through different topics related to youth work in the age of social media and mobile devices asking you to estimate if your competent for certain tasks. The idea of the quizzes is that you get a quick impression of your competence level by facing certain tasks. At the end of the quiz you will get a competence score. However, this is not an objective measurement. Therefor the ideas about your own professional development that evolve during the quiz are more beneficial than your score.

The quizzes are based on the work of the YOWOMO2.0 Partnership. However a major part of the phrases used in the quizzes is derived from the “The Social Media Resource Kit “ by Sun Sun Lim and Kakit Cheong, published by the Central Youth Guidance Office of the Ministry of Social and Family Services in Singapore. Dr Sun Sun Lim is Associate Professor at the Department of Communications and New Media at the National University of Singapore and provided consultancy support in the development of this quiz. The quizzes are available in English, Spanish, German and Turkish.

For a more summative assessment (at the end of the learning process) we chose the assessment of products which is described in the following section.

Assessments of products

Assessment of competences can be done using methodologies like observation, a portfolio of evidence, questioning, and products or processes. In YOWOMO2.0 we focus on products that can be used as evidence. Depending on the training situation other methods or combinations of methods may be affordable. For more information please go to “The Assessment Guide”.

When someone goes through a learning process that includes the performance of several tasks and that ends up in completing a product the product itself can be used as the evidence of knowledge, skills and attitude. Additional questioning and/or reporting may back up the knowledge and attitude components by asking for further information around the product.

A person is defined as competent when they are able to demonstrate that they can apply knowledge, skills and attitude successfully in their working environment.

In order to collect quality evidence you must ensure that you are sure of the competency requirements and thoroughly examine the required knowledge, skills and attitude as described the YOWOMO2.0 description of competences.

A good way to start the assessment process may be answering the following question looking at your training situation:

1. Expectations – What can you expect from a person at this level?
2. Application – Can the person apply what s/he has been learning?
3. Outcomes – What will be the outcomes?
4. Variable conditions – What range of contexts can the new skills be applied?
5. Range of transference – Can the new skills be transferred to new situations?

YOWOMO2.0 Competences & Products

The following list shows examples of products to be used as learning outcomes in trainings based on the YOWOMO2.0 description of competences. The main ideas of this list is to inspire the thinking about learning outcomes both in trainers and trainees. This is not a complete list because – depending on your training situation – other products may be more suitable. The list is structured according to the YOWOMO2.0 Competence Model:

Competence: Target groups

Description of target groups

• Describe at least three different target groups (possible) for your work. At least one of them shall be a group that you have difficulties to reach today.
• Describe what you know about them, what you believe that they need and wish.
• Show where and how you think you can get in touch with them and how you can use social media and/ or smartphones in this work. What type of platform and communication is suitable? (Chat, page, group, forum, information, promotion of activities etc.)
• Create with your target group a list of “do’s and don’ts” to show you have insight in their needs
• Demonstrate that you have made contact with a target group, related to your work, using online media.

Competence: Relations with clients

Description of relation with clients

• Make and show a professional presentation where it is possible to see on which terms/conditions you are on social media
• Make a online promotion for an activity or service that you are offering to your target group
• Describe what is important when communicating/building a good relations on social media or mobile devices in general and how to handle e.g. friendship requests
• Show how to handle a case – e.g. if you discover “some kind of abuse or Dysfunctional behaviour. What to do and when to hand over to another profession.
• In an interview, let the student/social worker speak about the relations with the target group. Let them explain what his/her role as a facilitator is on social media in working with clients.
• Demonstrate your online relation with clients by showing a portfolio of your (anonymised) relations with clients. (e.g. chats, Facebook, …)

Competence: Social Media and Mobile Devices

Online representation/profile according to the protocol of your organisation (e.g. security setting)

• that is “state of the art” (quality, function, credibility, reliability, attractiveness, systematic structure and navigation) and working cross-platform (e.g. using different media)
• that shows the organisation you are working for (e.g. logo, corporate identity)
  that shows the services you offer
• that explains the procedures and proper use in an easy way
• that offers information about legal and ethical issues (e.g. disclaimer, managing conflicts, privacy)
• that shows different youth work related interaction with clients

and a reflection about the experience of creating and using it

Suggest additional use of social media and mobile devices to support youth workers in their job.

Make a portfolio of your personal use of social media and mobile devices in your day-to-day job.

Competence: Dysfunctional behaviour

See relations! (Don’t focus too much on dysfunctional behaviour. It is just a way of looking to people. Concentrate more on things that they do right, and put that in the picture.

Competence: Protocols

Summarise the protocols that are used within your organisation and make suggestions to improve them or better implement them.

Description of when and how to use a protocol/logbook.

Description of reference guides, repository systems, rules and guidelines

• e.g. Safety planner, privacy, legislation (http://www.360safe.org.uk/Files/Documents/School-E-SafetyV3)
• electronic forms
• forms for résumés
• content of protocols
• part of job description
• local working plan

Let them co-create a collaboration plan with target group to work on safety.

Competence: Quality management

Descriptions or examples of

• Quality management system available
• Working diaries for online work
• Evaluation system
• Online questionnaires for evaluation, preparation
• Forum to register problems
• Print screens of the work, (Facebook page, chat conversation…)

Indicators

• Is it available?
• How often is it used?
• Who uses it?
• Is it communicated and how?
• Is it part of a functional evaluation of the staff

Show how the online media are a full part of the evaluation system in your organisation. Make suggestions for improvement.

Competence: Professional network and representing the organisation

Products for representing the organization online and showing the multidisciplinary network (for example)

• A social media profile (e.g. Facebook or Twitter) to represent the organization.
  A blog to represent the organization.
• An internal/closed forum for the staff of the organization to ensure contact and communication for administrative issues.
• An internal/closed forum (for the privacy) for the target group to ensure contact and communication.
• Membership/subscription to related online forums to get connections, discuss issues and collaborate with different organizations.
• Organizing events in collaboration with other organizations and/or support them.
• Designing documents e.g. brochures, flyers etc. to promote the organization and disseminating them online (through social media) and/or written press.
• Evaluating through feedback from the target group and stakeholders to improve the representation process. (Quality management)
• Working/collaborating with experts in different but related fields. Show a contact list over this.

Competence: Legal framework and ethical issues

As an indicators for legal and ethical issues the product has to be a document (contract) with all the needs and legal items to work with social media and mobile devices. in the document there have to be two major topics:

• A general topic with the explanation
• A specific topic for work in social work or in your field of work.

The specific and general topics are different for each country.

Ethical issues:

• Show how you interact with youth workers and young people in the discussion of ethical dilemmas.
• The youth worker has to search in the organisation for legal and ethical documents in the youth work specific and not specific to social media.
• As a youth worker you need to have a document where it is written in easy language what laws and the ethical issues are and where you can find the original texts.
• The document has to be online and accessible for everyone who has concern the legal and ethical issues:
  • management,
  • staff,
  • youth workers,
  • youth
  • and all in an easy language.
• Plan for reviewing the ethical and legal topics and how to keep them up to date.

Assessment criteria

As an assessor, you have to be confident that the assessment decision you make is based upon quality evidence. When you are reviewing evidence, think about the following quality criteria (for more details: The Assessment Guide):

Is it valid?

• Does the evidence relate to a unit of competence?
• Does the evidence reflect the four dimensions of competency?
• Does the evidence address the key competencies?

Is it sufficient?

• Does the evidence cover the full range of performance identified in the unit of competency?
• Does the evidence show competence over a period of time?
• Does the evidence show competence in a range of contexts?

Is it curent?

• Does the evidence show that the candidate can currently perform the competence while working?
• Is the evidence benchmarked against industry standards?

Is it authentic?

• Is the evidence the candidate’s own work?
• Are the qualifications, references and licences presented by the candidate authentic documents?

Example

Developing social media policies for youth work as part of social work study

Kristin is a fourth year social work student at Saxion University of Applied Sciences in Enschede/Netherlands. As she is running a part-time study internships are a major part of her study. During the YOWOMO2.0 training she developed a self-directed learning project to acquire the competence “Social media and mobile devices”. To assess her non-formal learning and competence-building she developed “A safety handling with social media and mobile devices” for the youth work organization where she is working during her part time study. She decided to develop two policies: one for the clients and one for the professionals. At the end of her learning project she was asked to do a self-assessment of her competence-building by comparing the outcomes (the policies) to the competence (Social media and mobile devices) using the criteria named above (valid, sufficient, current, authentic). Furthermore she was asked to use a grading system of 25 points per criteria which allows for gaining of a total of 100 points for her learning project.

Here Kristin’s self-assessment report:

In the end the results of the project “A safety handling with social media and mobile devices” were two policies. The first one is for the clients and shows how to use social media and mobile devices. The second policy is for the social worker and shows how to use social media at work.

The qualities of the policies are based on following criteria:

The policy for the social worker is acceptable for my colleagues.

The colleagues sign the policy, so it becomes a contract.

The policy for the clients is written together with a fifteen year old girl.

The girl and other kids sign the policy, so it becomes a contract.

The policy can be transferred into other stationary groups.

The product is valid, because the policy is related to the unit “Social media and mobile devices” of the competence frame and it reflects the four dimensions of competency. For example the job was performed in the required standard, because I did the project in a role of a social worker and it was integrated in team meetings. If I suppose that there are 25 points for this criterion, I would give it 25, because I have achieved all points.

The project is also sufficient. I reached a lot of competences on producing level and have a lot of knowledge about social media and mobile devices, which I can transfer into the work with the clients. I am satisfied with my learning process.

If I suppose that there are 25 points for this criterion, I would give it 20, because I haven’t acquired all competences.

The project is current, because it is a current topic in the practice. My colleague (the group manager) said: “The project was very current. Before the project I didn’t know anything about social media and didn’t know how to use it in the social work. Now I know it and can use it.”

If I suppose that there are 25 points for this criterion, I would give it 25.

The project is authentic, because it was my idea of the project. The evidence is my own work and it is suitable with my practical work.

If I suppose that there are 25 points for this criterion, I would give it 18, because there are a lot of policies for social media on the internet.

In the end the project has 88 of 100 points. I am satisfied with the learning process and I would like to learn more. The development of social media is fast and if the younger children would like to use social media and mobile devices there will be other technical standards. So Life Long Learning is an important aspect.

The self-assessment Kristin is an example for an assessment of a non-formal self-guided competence building which can be used for a shared decision making if there is a need for a kind of certification. In the case of Kristin her project was related to the Minor Child and Youth Welfare and she could use her project in the module “Professional Development” gaining ECTS for her social work study.