Pagalguy spoke to Dr Jitendra K Das, who is director at FORE School of Management and more importantly a former CAT paper -setter. Below he has not only disclosed important information on the normalisation process and on how CAT papers are set but also discussed the overdependence of students on coaching classes to get them through.
When were you actively associated with setting of CAT papers and for how long?
About 5 years back. And did it for 4-5 years.
What were your commitments while setting the paper. What did you have to keep in mind?
Setting of questions are done keeping in mind the categories and ways and means to capture the intellect of the student. Each faculty involved for setting papers at IIM develops questions in his or her own respective category, which goes to another faculty in a similar field, who also verifies whether the questions are eligible and acceptable. Selections happen by peers and everyone checks each other’s questions. And this way, in between the faculties, the questions are checked, validated and accepted. Once the question is accepted, it goes through a second round of checks whether the level of difficulty is assigned – like low, moderate or high.Thus the question bank is prepared in each category consisting of 100 or 150 questions.
Then the 50 questions are shortlisted and time required to solve the question is set depending on the difficulty level assigned by the faculty. This way the question bank is frozen and four sets are prepared with each set having a code on the top. Students were suppose to write the code as the answers depend on the code. That is how it used to be. Till the time different coaching centres came into the picture where students were taught how to “crack” the test.This is when the faculty realised that they had to be one step ahead of the coaching institutes.
Do you think it is practically possible to set thousands of different CAT papers in one CAT season?
Sure. I will will give you a hypothetical situation which we were actually considering in early 2000 where you can have one hundred thousand questions in one section. For example, lets say the analytical set having a question bank of 1 lakh question which can be classified under four difficulty levels ABCD. A candidate going for a online exam would be presented a mix of fixed number of questions from these difficulty levels which are at parity with each other. In fact that is the way to be done. No other choice. A question bank has to be there from where randomly questions can be picked and presented to students.
The normalisation process – does it not work against the students say in a slot if there are very few students?
See the normalisation has to be done because the difficulty level given to student may have varied. It is expected when there is a group of students on an average they will perform in the same level. Normalisation is done to scale up the low performing students group and to scale down high performing group – this how normalisation works where all students are put together at par with each other. And the number of students are never in small groups, its done in large group at an aggregate level and at PAN India basis kind of a thing. Hence can never be a disadvantage for them.
What is your opinion about the computerised version, do you see it as an advantage or disadvantage?
The need for computerisation was a natural progression due to the growth in numbers of the CAT aspirants. In the year 2000 I think there were only 65,000 candidates which gradually increased over the years and peaked to 3 lakh at some time. The paper pen examination mode posed a huge logistics issue where all the various venues across the country needed to be set in a manner, format and ambiance where the exams could be conducted with absolute fairness of the system and avoiding any untoward thing happening. It was a humongous task.
So discussions started happening and online was the only option. I had been advocating this personally but at that point of time the idea was not acceptable by many. There were other similar tests happening in the world like GRE and GMAT so it was not that a difficult task at it seemed to many. It was delayed and took a little time but finally the online test was carried in 2009. Some operational and technical glitches happened though probably which could have been avoided. This time things are looking good and students are enjoying best part being one can go back and change the answer.
But, computerisation is definitely advantageous in all respects. But then one objective of CAT has been compromised. When IIMs thought of getting one step ahead of coaching institutes, the concept of variable marks were introduced few years back wherein certain questions were allotted higher marks as per the intelligent quotient of the question which challenged the merit and the intellect of the student where one has to do may be certain calculations to get to the answer. Additional space were provided in those questions for the students to do the extra work required. The whole idea was that the questions must be asked in a such a way that it should trigger some thoughts process in the mind of the student.
And the response to the question would reflect the student’s thinking process and where the coaching institute cannot brain-train the students to get to the correct answer through short cuts. Until 2008 that process were followed.
How have CAT papers changed over the years
It has not changed much substantially over a decade or so. Traditionally questions were made keeping basic CAT objectives 3 modules in mind. Only in the early 2000 there was a effort made such that the coaching institutes could not sort of brain- train the students. Difficulty levels and patterns may have varied a little otherwise they remained almost similar in types. Questions were of equal marks traditionally but as I said few years back there were some changes made to this with the concept of variable marking system being introduced in order to measure the thought process of the student and to check his intellect. With this objective the format may have changed a bit.
How have students changed over the year?
Students have changed substantially over the years. If you go back few years, you will find students were extremely hardworking, spent long hours studying, trying to gather and acquire knowledge over the subject.Earlier CAT used to be a secret thing. It has changed from the days when question papers were collected back by the examination hall till a decade or so ago when it was decided that students could keep the papers – and this brought about considerable changes.
Earlier days, students used to focus on learning and gaining solid knowledge and practising it in terms of understanding and doing. Now, coaching institutes do not encourage any hair splitting analysis of the questions and are more into teaching how to find out the correct answer, without actually understanding the question. In fact, students are also trained on the probability of getting a wrong answer and how many wrong answer can one take without seriously jeopardising the chances of getting a good score.
So in all this analysis happening, students are also getting trained on how one can quickly reach the proficiency level of ‘cracking’ the CAT exam.
Do you think students lost out last year when so many glitches hap in the first version of the computerised paper?
It was rather an advantage for the student as CAT has been very proficient in calling those students and asking them to re-appear for the exams.
Do you think CAT scores should be given importance during admission, the GDPI or past academic record or work experience
You see, different institutes have different criteria for admission, including IIMs. All IIMs have a different approach and procedure and the weightage and criteria varies. I feel CAT score should be only one criteria as in real job situation there are other components or qualities which are looked into as well which are not really captured in the CAT exams. We are looking at other things like marks in post graduation, performance in GD, performances at the interview, overall qualities and competencies the students may have which can go as support to the student’s candidature for a business manager. Work-ex also plays an important role.
There are lots of do’s and dont’s while taking the CAT today, no toilet break for some time, no taking in a handkerchief. You think all the little rules are important?
Off course they are important to maintain discipline specially in an era where the technology is changing, so one has to maintain discipline as smarter means are available to todays generation. But, there has to be exceptions for example medical and health related cases which Im sure the CAT would take care off.
Should CAT be more transparent in its marking system
There has to be certain confidentiality maintained but on the negative marking account, a little more transparency on communicating actually what affects the students can be thought off in the lines of the factors involved in negative marking. Some secrecy has to be there – this is an exam.
On what is the objective of the confidentiality.
Students should not trade off on one question against other in terms of their ability to give a correct answer.
(Besides, setting CAT papers, Dr Jitendra Das was also faculty at various b-schools and taught Customer Relationship Management, Advanced Marketing Research, Marketing Management, Internet Marketing, particularly at IIMA, IIML, SP Jain Management Center, Singapore, IIM Kozhikode and IIM Ahmedabad. He has also been with the Indian Institute of Forest Management, Bhopal (IIFM).