“In the world of psychology, there is a view that a correlation exists between personality traits and performance (behavior). After some research, we were also able to make some sense out of the notion that behavior could correlate to values. However, we struggled to understand the correlation between values and personality.”

Assessing Values in Online Technology

Ten Things We Learned

Part Two

This article and study was co-authored by Zachary Harper, Tejal Raval, Anum Malik, and Michael Kannisto.

Ten things we learned – Summary

We covered the abstract and methodology in the first article in this series and today we’ll be looking at a summary of the ten things we learned.

Here’s where we are in the series:

• Assessing Values in Online Technology – Part 1
• Assessing Values in Online Technology – Part 2
• Assessing Values in Online Technology – Part 3
• Assessing Values in Online Technology – Part 4

Salespeople are Not I/O Psychologists

Salespeople are paid to sell a product; consequently, they may only be well versed in their sales pitch and not the specifics of a product. If they do not offer a discussion with a product specialist, then ask for one. Keep an eye out for data points (i.e. correlation coefficients) thrown around without context, as this may indicate the salesperson does not understand the data.

Accuracy, Validity, and Reliability are All Equally Important

Salespeople are paid to sell a product; consequently, they may only be well versed in their sales pitch and not the specifics of a product. If they do not offer a discussion with a product specialist, then ask for one.

Ask for a technical manual that supports an assessment’s science with research. Be wary of companies that do not have any kind of technical document or research support.

Ensure International Assessment Companies are Compliant

Assessments that are compliant in one country may be non-compliant in another, so it is important to ensure that international companies are compliant in all countries where a business operates. This could include indicating compliance to the GDPR (UK), ISO (Int’l), and EEOC (US), as well as the ADA (US) and WCAG (Int’l).

A Longer Duration Does Not Necessarily Indicate a Better Assessment

Assessment results are not designed to serve as the sole indicator for hiring decisions; they have limited utility. An assessment that is twice as long as another will not necessarily give you twice as accurate results, so you will have to weigh the impact on the candidate experience against the assessment duration and the value of the output.

Companies Offering Custom Branding are Doing It Right

Some companies offer space for a custom video or photos before or after the assessment; these custom branding techniques can have a variety of uses, from providing instructions for how to take the assessment to showing a candidate around the office to prepare them to come in for an interview. Whatever the case, we found that custom branding helps a candidate get to know a company’s culture better, ultimately encouraging them to continue in the application process.

Many Assessments are Offshoots of the Same Model

A majority of the time, an assessment is based off of the Five-Factor Model of Personality (AKA The Big Five), as it is widely used and accepted in the psychology world. Assessment companies may try to extrapolate this model to tailor the assessment to specific needs, but they should have empirical research to back this up.

True Values-Based Assessments are Scarce

An assessment that is twice as long as another will not necessarily give you twice as accurate results, so you will have to weigh the impact on the candidate experience against the assessment duration and the value of the output.

There are only a handful of solely values-based assessments on the market, so assessment providers may offer mapping specific workplace behaviors to desired values. This sparks debate, though, because the individual mapping the behaviors has to be qualified to do so, and personality traits and workplace behaviors do not directly influence or indicate a person’s values.

Generally, Up-and-Coming Companies will be More Forward

It was easy to discern which companies were new in the assessment space, as they were the ones emailing and calling, often weekly, to ask for an update on the project. The assessment providers that had been in the space for longer only checked in once a month or so.
Both of these styles have their benefits and drawbacks, so be prepared for both.

If a Company Doesn’t Believe in Cultural Variation, Run

If a company does not believe cultural variation will affect an assessment’s output or candidate’s score, it may be best to move on. Different regions, whether in the same country or abroad, are going to have different workplace behaviors. Therefore, there should be a way to change benchmarks or weigh different values more or less to tailor it to a region.

Bespoke Assessments are Becoming More Abundant

Just under 20% of the assessment providers that we reached out to provide completely bespoke assessments. This made gathering specific information difficult, as they could not answer some of the follow up questions without creating the entire assessment for us. Although bespoke assessments boast a more individualized candidate experience, it is worth noting the cost of implementing such an assessment versus the value the assessment will truly provide to the candidate experience.

Discussion

We discovered early on the importance of understanding the distinction between values, personality, and behavior. Personality is what you are. Values define who you are. Behavior drives what you do.

In the world of psychology, there is a view that a correlation exists between personality traits and performance (behavior). This connection has a long history in the literature, and forms the basis of the assessment industry. After some research, we were also able to make some sense out of the notion that behavior could correlate to values. However, we struggled to understand the correlation between values and personality. During the course of our conversations, we noticed that many vendors used the terms personality, behavior and values interchangeably.

Our conclusion is that not many tools truly assess values directly. Many assess personality traits or behaviors and then attempt to correlate them to values.

Stop back soon for part three in our series on Assessing Values in Online Technology.

Authors

Tejai Raval Tejai Raval is a Talent Operations expert with experience in Fashion, Consumer Goods, and the Financial Industry. She has a Master’s Degree in Human Resources from Farleigh.

Zachary Harper Zachary Harper is a Master’s Student at Iowa State University and a recent graduate from Washington State University Tri-Cities. He received a B.A. in Business Administration with an emphasis in Human Resource Management as well as a Global Leadership Certificate.

Anum Mlik Anum Malik is an Organizational Development specialist. She has a Master’s Degree in Psychology from Harvard University.

Michael Kannisto Michael Kannisto is a Talent Acqusition professional with a strong interest in the Future of Work and Scenario Planning. He has a Ph.D. in Chemistry from Texas A&M University.