Creation

We began by doing in-house research to create a pool of candidate questions that tapped into different aspects of mental health and well-being. Our Head of Care, Ms. Smruti Deshpande (MA Clinical Psychology, Fergusson College; MPhil, NFSU), and People Scientist, Adwaita Deshmukh (MA Psychology, Pursuing PhD in I/O Psychology at Savitribai Phule Pune University) led this activity.

Ms. Mukta Dhavalikar, a Licensed Clinical Psychologist, Therapist & Founder of ManoVed, Goa, joined the team as a consultant to review the domains selected, the definitions, and the item pool.

We composed over 110 items (questions) based on prominent aspects of employee well-being outlined in classic psychological theories such as the PERMA model by Seligman (2011), and published work by Fisher (2014). We took a full-spectrum approach focused not just on the absence of negatives but also on the presence of positives, which denoted thriving and flourishing. Each item was given suitable answer options while avoiding double negatives. We carefully avoided clubbing of more than one aspect in one item, and followed best practices of item writing according to the American Psychological Association’s Handbook of Testing and Assessment (2013).

We ran a pre-pilot (100+ participants) at this stage to study the normalcy of each item in the general population. Simultaneously, the entire item pool was sent for expert validation.

Validation

Content Validity

We consulted various experts during different stages of the creation of the index.

As part of the content validation process, the following experts validated the item pool:

1. Ms. Sharmin Palsetia

Assistant Professor, Dept of Psychology, St. Mira’s College, Pune

Specialization – Industrial/Organizational Psychology

Extensive research and psychometric work experience, pursuing a doctorate to study dimensions of leadership and develop a measurement tool.

2. Ms. Radhika Bhasin

Deputy Manager, HR and Administration at Rieco Industries

Specialization – Industrial/ Organizational Psychology

Extensive experience in training and coaching employees, significant work done with psychometricians creating and validating psychometric assessments.

3. Ms. Pradnya Joshi

Manager of People Operations at One2N

Specialization – Industrial/Organizational Psychology

Extensive experience creating and validating psychometric tools and assessment centers end-to-end, both individually and with organizations in the past.

The experts rated all the items in the item pool on the following aspects, on a scale of 1 to 5.

Relevance: Measured whether the item goes with the description of the theme it belongs to.
Necessity: Measured whether removal of the item would make the tool less representative.
Lucidity: Measured whether an average corporate employee would understand the item.

The experts also reviewed the items for any grammatical errors or other problems, and suggested new aspects not covered in the item pool.

The ratings of all the Subject matter experts were then collated and averaged. Only those items finding a good average rating on all the aspects and having a normal response distribution in the pre-pilot were retained A few items suggested by the experts were added too.

Our next goal was to assess how well the Thought Sutra Index does its job. To do this, we invited another large sample of participants - a representative sample of the Indian working population - to complete the now-finished Index, along with some other established gold-standard mental health measures, including the General Well-Being Schedule.

Factorial Validity

To explore how the different domains fit together, we used a Statistical method called exploratory factor analysis. From this analysis emerged a tight, parsimonious and robust model of well-being that consisted of four domains, namely Physical Well-being, Calmness, Social Well-being and Meaning in Work. The questions included in the tool got reduced to a group of the most discerning questions, each question tapping into a different aspect of one of the four domains, and each domain, in turn, tapping into a different aspect of overall well-being. This final index form now has 27 questions, with a few additional probes to assess mood separately.

We used this model to build an algorithm for scoring users’ responses on the Thought Sutra Index.

Internal Consistency

An indicator of whether a psychometric tool is measuring the same construct and whether all of its questions are aligned in their target is the reliability index of the tool. We measured the internal consistency of the tool using the standard statistical method of computing Cronbach’s alpha coefficient for each domain and for the overall well-being scores. High internal consistency indicates a trustworthy tool with minimal margin of error and reliable results. All of the consistency coefficients were in the ideal range, as presented below:

Item Discrimination Index

In a scientific assessment of psychological attributes, it is important to ensure that each item is capable of discriminating well between individuals high on that attribute and individuals low on that attribute. For each item, the discrimination power is measured by computing the difference between average score of the high-scorers and average score of the low scorers on that item and dividing it by the total possible score. The discrimination indices for all items in the Thought Sutra Index ranged between 0.3 and 0.6, which indicate the most powerful discernment. Thus, the Index is proven to be a perceptive tool that detects the smallest of differences in well-being across individuals.

Criterion Validity

To ensure that questions in each domain measure what they’re intended to measure, we checked that domain scores correlated with corresponding well-established existing measures - what is known as concurrent validity, a type of criterion validity. For example, we found that Thought Sutra Index Calmness scores are strongly negatively correlated with scores on the Anxiety Subscale of the General Well-Being Schedule, a gold-standard measure of anxiety; and Meaning in Work scores are strongly positively correlated with the Utrecht Work Engagement Scale, the gold standard measure of vigour, dedication, and absorption at work. A detailed table below shows all of the concurrent validity coefficients and their statistical significance levels.

The criterion tests used for Validation:

**indicates significance at .01 level

*indicates significance at .05 level

Taken together, these results show that the Thought Sutra Index is a valid and reliable measure of overall mental well-being and of four key domains that make up well-being. We’ve demonstrated that it’s an appropriate tool to enable users to explore their own minds and to help direct them to the care most suited to their individual needs.

How to interpret the correlation coefficients for validity:

Correlation essentially measures the strength of relationship between two factors, e.g. when stress shows a strong positive correlation with severity of health issues, it means that as stress increases, severity of health issues also increases. A negative correlation, denoted by a negative coefficient, means that as one variable increases, the other variable decreases. The correlation coefficient denotes the relationship numerically, with 0 indicating no correlation, coefficients below 0.3 indicating a weak correlation, those between 0.4 to 0.6 showing a moderate correlation, and those at 0.7 and above showing a strong correlation.

Measuring psychological phenomena is much more complex than measuring physical properties such as length or weight, which do not change with circumstances or chance. So a component of error is unavoidable in measuring relationships between psychological factors. However, robust statistics have minimal chance of error, as indicated by significance levels of 0.01 or 0.05, meaning a less than 1% chance or less than 5% chance, respectively, that the computed coefficient is erroneous.

You may also be interested in

The Science behind the Thought Sutra Index