Our research

We have found that relying on already existing research is simply not enough to put forward our visionary tools and methods. We therefore conduct our own research activities in cooperation with several universities in the US and Germany:

Institut fÜr Theoretische Psychologie, UniversitÄt Bamberg
Georg-Elias-MÜller-Institut fÜr Psychologie, UniversitÄt GÖttingen
Department of Sociology, Kent State University

Research projects recently completed

Predicting job performance with the BQO

Using the AHS at IBM Germany leads to increased performance of new hires

A study conducted with IBM Germany on N=450 new hires hired between 1999 and 2003 shows a significant increase in job performance of newly hired employees after the AHS was implemented. The AHS was used for pre-selection of applicants as a first step in a 2-stage selection process comprised of the BQO and a 1-day Assessment Center.

After implementing the AHS, job performance of new hires as assessed through supervisor ratings 1 to 3 years after applicants have completed the BQO increased by 2/3 of a standard deviation. This increase is highly significant. Since all other components of the selection process remained the same, the increase in performance can clearly be attributed to the implementation of the AHS.

Results show that implementing the AHS as a pre-selection instrument leads to a substantial improvement in employee performance.

Scitrain study on correlations between behavior, biography, and personality

Study reveals substantial intercorrelations between behavioral, biographical, and personality predictors for job success

In this study, N=46 university graduates majoring in business administration at the university of Bamberg/Germany completed the Biographical Questionnaire Online and the Personality Questionnaire Online. Participants further took part in a 1-day state-of-the-art Assessment Center comprised of a computer-simulated business game, a role play, and a hiring interview.

Results show substantial correlations between Assessment Center scores, biographical indicators, and personality traits. The Advanced Hiring System predicted performance in the Assessment Center with R=.49. Career-related scales of the Scitrain Personality Inventory * such as Career Orientation (R=.47), Organizational Skills (R=.46) or Team Leadership (R=.44) also yielded substantial correlations with Assessment Center results. Furthermore, biographical predictors and personality traits showed substantial intercorrelations. International experience as measured through the BQO correlated with Team Leadership with R=.42, weekly activites (BQO) correlated with Organizational Skills (PQO) with R=.4.

This study demonstrates the potential of behavioral, biographical, and personality predictors for predicting job success. All three domains contain valid information for employee hiring.

Would you like to see more detailed results of the study? Check out the matrix of intercorrelations * between selected predictors.

Predicting Assessment Center scores with the PQO

Selected PQO profiles show high correlations with Assessment Center scores

In a study involving 150 applicants applying for a job with a large company, PQO profiles were correlated with the results of a 1-day Assessment Center. Results yielded substantial correlations between PQO-profiles and Assessment Center scores. The PQO profile “Manager” correlated with the overall AC score with R=.46, the PQO profile “Sales” showed a correlation of R=.58 with the AC score.

The results of this study show that using the PQO as a pre-selection instrument before an Assessment Center or an interview leads to a substantial improvement in the quality of hires considered in the Assessment Center or the interview.

Problem solving and decision making in teams: What makes teams successful?

Differences in performance of problem solving teams can clearly be attributed to differences in communicative behavior.

In a study on problem solving and decision making in teams, N=21 teams comprised of 4 to 6 people (practitioners, managers, and university students) played the computer-simulated business game Cosmetica over the course of a full day.

The goal of the study was to find out how successful and unsuccessful teams differ in their problem solving behavior. The study was conducted in order to test the integrative theory of problem solving and decision making in teams put forth by Stempfle (2004a, 2004b, 2005).

A behavioral analysis was conducted on the four best-performing as well as the four worst-performing teams as measured by business game results.

Verbal interaction in the teams was completely recorded and coded employing several different coding schemes.

Comparing the behavior of successful and unsuccessful teams revealed a number of important differences.

Successful teams uttered a significantly higher proportion of communicative acts aimed at setting goals, proposing solutions, and stating hypotheses whereas unsuccessful teams limited themselves to exchanging more information.
Successful teams displayed more constructive criticism, whereas unsuccessful teams tended to utter more acts of agreement.
Successful teams planned more in early stages of groupwork, whereas unsuccessful teams displayed a more equal distribution of planning activities over the course of the problem solving process.
Successful teams were better able to cope with setbacks, mistakes, and crisis, as displayed by a significantly lower proportion of communicative acts concerned with regulating socio-emotional processes.
Successful teams divided their action into an initial orientation- and planning and a subsequent action stage. They avoided premature decisions, but emphasized gathering information, forming hypotheses, and planning before taking subsequent action. On the contrary, in unsuccessful groups decisions were distributed evenly across the problem solving process.
Successful teams flexibly adjusted their communicative behavior to the demands of the problem at hand, whereas unsuccessful teams showed less variation in their communicative behavior depending on different types of problem.
Successful teams were more critical of their own group process and deliberately changed group structures and processes repeatedly, whereas unsuccessful teams showed little self-critical behavior and few efforts to change their proceeding.

The results of this study point to the importance of communicative competence for team problem solving and decision making. They provide an excellent basis for subsequent research on training problem solving and decision making teams. *

Current research projects

Operational intelligence

Researchers have long argued that cognitive traits and competences such as general intelligence are good predictors for job success in various domains. Practitioners, however, have long criticized this position, pointing to the importance of alternative competencies such as social skills, personal competencies, or emotional intelligence for job success. Our own research has repeatedly yielded substantial correlations between cognitive predictors such as grades or intelligence and educational performance, but low or no correlations between such cognitive predictors and job performance.

Doerner (1989) has introduced an alternative concept of intelligence termed operational intelligence. By operational intelligence Doerner refers to competence in solving complex tasks characterized by goal conflicts, a large amount of problem elements, interconnectedness between these problem elements, and (partial) intransparency. Doerner argues that operational intelligence can not be measured by traditional intelligence tests, but must be assessed through complex dynamic problem solving scenarios.

The concept of operational intelligence has proven fruitful with practitioners in HR diagnostics, particularly in Europe. The construct of operational intelligence bears close ties to competencies such as strategic thinking, problem solving, or decision making that are included in many competency models and job profiles used across various companies and industries. Complex problem solving scenarios have thus been used as a tool for assessing operational intelligence in Assessment Centers. *

However, up to this date the concept of operational intelligence is still not particularly well-defined. The specific competencies and behaviors that operational intelligence is made up of have not yet been identified, and there is no agreed-upon definition of operational intelligence.

With this research project, we pursue three goals:

Establish a theory of operational intelligence.
Create measurement instruments for measuring operational intelligence accurately, reliably, and economically.
Explore how operational intelligence is linked to personality, behavior, and performance on the job.

This research project will be completed in 2008.

Enhancing the performance of decision making teams

Evaluating the effect of alternative training methods on the performance of decision making teams

In a joint study conducted by scitrain, the university of Goettingen/Germany, and the university of Bamberg/Germany, we research the effect of alternative training methods on decision making teams.

Teams of four will be assigned to three different training conditions and a control condition. All teams will play multiple scenarios of two computer-simulated problem solving scenarios simulating realistic business environments.

While playing the first scenario, a training intervention will be administered to each team in the three training conditions. Training interventions will be embedded in a problem solving scenario.

Teams in condition 1 will receive verbal instructions on team problem solving and decision making.
Teams in condition 2 will receive feedback concerning their behavior in the problem solving scenario.
Teams in condition 3 will be prompted to self-reflect on their behavior in the problem solving scenario.

Teams in the control condition will play the problem solving scenario without receiving any training.

The effect of the interventions will be assessed by comparing the performance of the teams in two subsequent problem solving scenarios, one related to the problem solving scenario played before and one structurally different from the previous problem solving scenario.

The study will provide answers to the following questions:

Can team problem solving and decision making competence be enhanced through training?
How effective are alternative training methods for enhancing decision making performance?
Can gains in decision making competence be transferred to a different problem solving task?

This research project will be completed in 2007.


Home Advanced Hiring System Leadership Potential Center Case Studies Scientific Background The Company Our Mission Our Management Team Our Partners Our Research Interns & Students Contact Us Downloads & Publications Please contact us at: 100 Lincoln Way East Suite 180 Massillon, Ohio 44646 Phone: 330.880.4888 Fax: 330.880.5029 info@scitrain.com	Our research We have found that relying on already existing research is simply not enough to put forward our visionary tools and methods. We therefore conduct our own research activities in cooperation with several universities in the US and Germany: Institut fÜr Theoretische Psychologie, UniversitÄt Bamberg Georg-Elias-MÜller-Institut fÜr Psychologie, UniversitÄt GÖttingen Department of Sociology, Kent State University Research projects recently completed Predicting job performance with the BQO Using the AHS at IBM Germany leads to increased performance of new hires A study conducted with IBM Germany on N=450 new hires hired between 1999 and 2003 shows a significant increase in job performance of newly hired employees after the AHS was implemented. The AHS was used for pre-selection of applicants as a first step in a 2-stage selection process comprised of the BQO and a 1-day Assessment Center. After implementing the AHS, job performance of new hires as assessed through supervisor ratings 1 to 3 years after applicants have completed the BQO increased by 2/3 of a standard deviation. This increase is highly significant. Since all other components of the selection process remained the same, the increase in performance can clearly be attributed to the implementation of the AHS. Results show that implementing the AHS as a pre-selection instrument leads to a substantial improvement in employee performance. Scitrain study on correlations between behavior, biography, and personality Study reveals substantial intercorrelations between behavioral, biographical, and personality predictors for job success In this study, N=46 university graduates majoring in business administration at the university of Bamberg/Germany completed the Biographical Questionnaire Online and the Personality Questionnaire Online. Participants further took part in a 1-day state-of-the-art Assessment Center comprised of a computer-simulated business game, a role play, and a hiring interview. Results show substantial correlations between Assessment Center scores, biographical indicators, and personality traits. The Advanced Hiring System predicted performance in the Assessment Center with R=.49. Career-related scales of the Scitrain Personality Inventory * such as Career Orientation (R=.47), Organizational Skills (R=.46) or Team Leadership (R=.44) also yielded substantial correlations with Assessment Center results. Furthermore, biographical predictors and personality traits showed substantial intercorrelations. International experience as measured through the BQO correlated with Team Leadership with R=.42, weekly activites (BQO) correlated with Organizational Skills (PQO) with R=.4. This study demonstrates the potential of behavioral, biographical, and personality predictors for predicting job success. All three domains contain valid information for employee hiring. Would you like to see more detailed results of the study? Check out the matrix of intercorrelations * between selected predictors. Predicting Assessment Center scores with the PQO Selected PQO profiles show high correlations with Assessment Center scores In a study involving 150 applicants applying for a job with a large company, PQO profiles were correlated with the results of a 1-day Assessment Center. Results yielded substantial correlations between PQO-profiles and Assessment Center scores. The PQO profile “Manager” correlated with the overall AC score with R=.46, the PQO profile “Sales” showed a correlation of R=.58 with the AC score. The results of this study show that using the PQO as a pre-selection instrument before an Assessment Center or an interview leads to a substantial improvement in the quality of hires considered in the Assessment Center or the interview. Problem solving and decision making in teams: What makes teams successful? Differences in performance of problem solving teams can clearly be attributed to differences in communicative behavior. In a study on problem solving and decision making in teams, N=21 teams comprised of 4 to 6 people (practitioners, managers, and university students) played the computer-simulated business game Cosmetica over the course of a full day. The goal of the study was to find out how successful and unsuccessful teams differ in their problem solving behavior. The study was conducted in order to test the integrative theory of problem solving and decision making in teams put forth by Stempfle (2004a, 2004b, 2005). A behavioral analysis was conducted on the four best-performing as well as the four worst-performing teams as measured by business game results. Verbal interaction in the teams was completely recorded and coded employing several different coding schemes. Comparing the behavior of successful and unsuccessful teams revealed a number of important differences. Successful teams uttered a significantly higher proportion of communicative acts aimed at setting goals, proposing solutions, and stating hypotheses whereas unsuccessful teams limited themselves to exchanging more information. Successful teams displayed more constructive criticism, whereas unsuccessful teams tended to utter more acts of agreement. Successful teams planned more in early stages of groupwork, whereas unsuccessful teams displayed a more equal distribution of planning activities over the course of the problem solving process. Successful teams were better able to cope with setbacks, mistakes, and crisis, as displayed by a significantly lower proportion of communicative acts concerned with regulating socio-emotional processes. Successful teams divided their action into an initial orientation- and planning and a subsequent action stage. They avoided premature decisions, but emphasized gathering information, forming hypotheses, and planning before taking subsequent action. On the contrary, in unsuccessful groups decisions were distributed evenly across the problem solving process. Successful teams flexibly adjusted their communicative behavior to the demands of the problem at hand, whereas unsuccessful teams showed less variation in their communicative behavior depending on different types of problem. Successful teams were more critical of their own group process and deliberately changed group structures and processes repeatedly, whereas unsuccessful teams showed little self-critical behavior and few efforts to change their proceeding. The results of this study point to the importance of communicative competence for team problem solving and decision making. They provide an excellent basis for subsequent research on training problem solving and decision making teams. * Current research projects Operational intelligence Researchers have long argued that cognitive traits and competences such as general intelligence are good predictors for job success in various domains. Practitioners, however, have long criticized this position, pointing to the importance of alternative competencies such as social skills, personal competencies, or emotional intelligence for job success. Our own research has repeatedly yielded substantial correlations between cognitive predictors such as grades or intelligence and educational performance, but low or no correlations between such cognitive predictors and job performance. Doerner (1989) has introduced an alternative concept of intelligence termed operational intelligence. By operational intelligence Doerner refers to competence in solving complex tasks characterized by goal conflicts, a large amount of problem elements, interconnectedness between these problem elements, and (partial) intransparency. Doerner argues that operational intelligence can not be measured by traditional intelligence tests, but must be assessed through complex dynamic problem solving scenarios. The concept of operational intelligence has proven fruitful with practitioners in HR diagnostics, particularly in Europe. The construct of operational intelligence bears close ties to competencies such as strategic thinking, problem solving, or decision making that are included in many competency models and job profiles used across various companies and industries. Complex problem solving scenarios have thus been used as a tool for assessing operational intelligence in Assessment Centers. * However, up to this date the concept of operational intelligence is still not particularly well-defined. The specific competencies and behaviors that operational intelligence is made up of have not yet been identified, and there is no agreed-upon definition of operational intelligence. With this research project, we pursue three goals: Establish a theory of operational intelligence. Create measurement instruments for measuring operational intelligence accurately, reliably, and economically. Explore how operational intelligence is linked to personality, behavior, and performance on the job. This research project will be completed in 2008. Enhancing the performance of decision making teams Evaluating the effect of alternative training methods on the performance of decision making teams In a joint study conducted by scitrain, the university of Goettingen/Germany, and the university of Bamberg/Germany, we research the effect of alternative training methods on decision making teams. Teams of four will be assigned to three different training conditions and a control condition. All teams will play multiple scenarios of two computer-simulated problem solving scenarios simulating realistic business environments. While playing the first scenario, a training intervention will be administered to each team in the three training conditions. Training interventions will be embedded in a problem solving scenario. Teams in condition 1 will receive verbal instructions on team problem solving and decision making. Teams in condition 2 will receive feedback concerning their behavior in the problem solving scenario. Teams in condition 3 will be prompted to self-reflect on their behavior in the problem solving scenario. Teams in the control condition will play the problem solving scenario without receiving any training. The effect of the interventions will be assessed by comparing the performance of the teams in two subsequent problem solving scenarios, one related to the problem solving scenario played before and one structurally different from the previous problem solving scenario. The study will provide answers to the following questions: Can team problem solving and decision making competence be enhanced through training? How effective are alternative training methods for enhancing decision making performance? Can gains in decision making competence be transferred to a different problem solving task? This research project will be completed in 2007.