I would argue that in person exams with no resources to do research goes against how the world works for most white collar workers.
Few are unable to research on the internet to verify information, or at least look at say a man page for coding or look up past stuff on stackoverflow, if they are working through a problem.
Standardized testing is just not as useful as-is. I do great at it and can typically pass exams without really studying the material, but others are not so lucky.
I’ve met people who can flunk exams but talk about the problems, go into how they would fix it, and work through a problem to implementation and testing in the real world.
Oh, and LLMs are the new typewriter, for better or worse. It’s unlikely we are going to have a future where they are not readily available. We already have models that run locally and do not transmit data anywhere, and AI customized to your own data that is not shared is already a service provided by Microsoft.
Education needs to evolve with technology. It’s always been 5-10 years behind the curve.
Maybe we should be using LLMs to proctor tests and generate interactive testing. Grading can be verified by a professor reading a transcript to verify hallucinations didn’t occur or influence the results. We can even have LLMs monitor the working process of people to help determine what are the most efficient ways to work custom tailored to individuals. This is just one idea of many potential options.
Those are all very nice ideas, and we’ll see if they pan out in the future. But universities need ways to stop (or, fine, reduce) cheating that can be implemented right now. A class in English literature and composition should test how well you can read and interpret the source material to then express something about it in your own words in a coherent way. This is a useful life skill to have, and students should learn to do it without AI assistance. Giving them a pen and paper and a quiet room to work in has been a good enough method of assessment for at least the last 50 years which is reasonably cost effective.
Yes, there are problems with standardized testing. Yes, you can cheat on a paper test. But the way to improve the evaluation process is to first establish a stable baseline, and then try new things that might work better to see if they actually work better. Not to throw out everything we knew before and haphazardly try every random idea that pops into someone’s head in a panic.
perfect not being the enemy of the good and all that
I would argue that in person exams with no resources to do research goes against how the world works for most white collar workers.
Few are unable to research on the internet to verify information, or at least look at say a man page for coding or look up past stuff on stackoverflow, if they are working through a problem.
Standardized testing is just not as useful as-is. I do great at it and can typically pass exams without really studying the material, but others are not so lucky.
I’ve met people who can flunk exams but talk about the problems, go into how they would fix it, and work through a problem to implementation and testing in the real world.
Oh, and LLMs are the new typewriter, for better or worse. It’s unlikely we are going to have a future where they are not readily available. We already have models that run locally and do not transmit data anywhere, and AI customized to your own data that is not shared is already a service provided by Microsoft.
Education needs to evolve with technology. It’s always been 5-10 years behind the curve.
Maybe we should be using LLMs to proctor tests and generate interactive testing. Grading can be verified by a professor reading a transcript to verify hallucinations didn’t occur or influence the results. We can even have LLMs monitor the working process of people to help determine what are the most efficient ways to work custom tailored to individuals. This is just one idea of many potential options.
Those are all very nice ideas, and we’ll see if they pan out in the future. But universities need ways to stop (or, fine, reduce) cheating that can be implemented right now. A class in English literature and composition should test how well you can read and interpret the source material to then express something about it in your own words in a coherent way. This is a useful life skill to have, and students should learn to do it without AI assistance. Giving them a pen and paper and a quiet room to work in has been a good enough method of assessment for at least the last 50 years which is reasonably cost effective.
Yes, there are problems with standardized testing. Yes, you can cheat on a paper test. But the way to improve the evaluation process is to first establish a stable baseline, and then try new things that might work better to see if they actually work better. Not to throw out everything we knew before and haphazardly try every random idea that pops into someone’s head in a panic.