Are the SAT tests seriously flawed?

The tests do have their problems, but I don’t think they are so easily gamed by the test-taking population at large. The ones that are able to understand and apply the code-cracking principles are usually capable of understanding the materials. Why tests have the weird psychology in their construction is intriguing.

My daughter took a prep class for the LSAT, and the program was about recognizing question types and how to approach the correct answer. While she didn’t ace the test, she did well enough to be accepted to two law schools at age 19. (Oddly, 19 year olds in law school is more common than you would think.)

Perhaps the tests are less about the subject matter and more about measuring abstract thinking and reasoning. Students are often more dominant in one of either language and mathematical analytics, and the test prep programs seem to help equalize the weaker ability.

I was an SAT tutor and SAT-course instructor for four years.

Having seen diverse levels of skill and diverse scores, I think the tests are good for what they are. Sure, they’re flawed, but most admissions committees recognize their flaws.

From as best as I can tell, SAT scores are a measure of how easily a student “gets it.” It doesn’t measure knowledge, but it seems to be highly correlated with how easily a student is able to understand new knowledge. That’s an important quality for any academic pursuit.

They are called SAT reasoning tests for, well, a reason. They are intended to test your ability to employ problem-solving strategies rather than to test your knowledge of the subject. That is what the SAT IIs, the subject tests, are for. However, the tests are much too formulaic for them to be a reasonable test of this ability, which is indicated by the fact that it is even possible to take a course to prepare for them. Taking such a preparatory course is, theoretically, cheating, so long as the purpose of the test is to determine your ability to solve unfamiliar types of problems. However, it would be impossible to eradicate such courses, and because of this, you are doing nothing especially wrong in teaching them, because to deny the same advantage to one student than another can get elsewhere would be unfair. The SAT would do much better to change its format to include fewer, but longer questions, that require that one sort through information presented in a creative and confusing manner, that varies from problem to problem and from test to test such that preparatory courses are useless. Only then will the test be an accurate reflection of ability. Of course, the CollegeBoard has no motivation to change, because the tests are already so wildly accepted, and they make so much money off of preparatory materials.

SATs are more of aptitude than achievement tests, and i think time is a huge factor in it. i’m not praising the test by any means (i’m taking it for the first time in march), but it’s a lot more time efficient to have multiple choice. the tests have about a .5 (out of 1) ranking of effectiveness in what they’re supposed to do, i think, and it’s just the best they have right now. they’re decent at doing what they’re supposed to do, i guess.

For the SATs to be so easily gamed, does this mean that there is something very wrong with their design?

Yes. It is very unclear to me what they measure, other than you belong to a class similar to that of those who design the test. Knowledge and aptitude are culturally determined. High achievement among children is most highly correlated with the highest academic achievement of the father. Of course, the same group also has the highest incomes. So, if you have money, you’re smart, or, if you’re smart, you have money.

The opposite is true of people who are poor and don’t do well on tests.

I’m sure we could write tests that people of lower classes would do better on that people from higher classes. That won’t happen, though, because upper class folks have all the power in society.

So the tests probably efficiently measure socio-economic status as much as anything else. Since wealthier people are more likely to be able to afford prep courses, the prep courses primarily reinforce existing patterns.

Do I compromise my ethics by teaching such a course, or can I justify what I do by saying that I am helping to force the ETS to reevaluate their way of doing things?

What you want to tell ETS is not news to them. They will only change if higher education institutions stop using them to evaluate students. There are a few competing tests, such as the ACT, and some institutions are dispensing with tests and going to other, more individualized methods of evaluating prospective students. But the SAT is still king, and most colleges just don’t have enough money to run admissions departments that can actually conduct individual evaluations of all applicants. In the absence of that, the SAT remains secure in its role in the process.

If you want to change the system, you should work in a college admissions office, learn a lot, develop a new, cost-effective system that does not use standardized tests, and eliminates various prejudices, and corrects for class bias. Sell that system everywhere, and the SAT will become history.

Other than that, feel free to do the job. Remember, you are actually teaching something valuable. Test taking skills are applicable in other areas, and they do teach people how culture and class work. If you can deconstruct the “head” or a culture, you’ve got a leg up. Prep courses help young folks do that kind of deconstruction.

The economic divide you bring up is certainly an issue. Another factor is cultural, favoring not so much whites in general as immigrants from Asian and European countries. These societies not only stress academic excellence in their children, but are also heavily built around testing. In France, for instance, there is the Baccalaureate, which essentially determines what careers are open to you, while in Japan the massive suicide rate is in part traced to the incredible amount of stress placed on students by the high-stakes exams they must take. Because they are used to such an environment, many immigrant families will have their children begin preparations for the SAT and other standardized tests at a ridiculously early age, often in middle school. This adds an extra layer of separation from the original goal of testing the problem-solving ability of students, as there is a whole subset of the population whose philosophy towards standardized testing entails intensive training for each specific exam.

From what @daloon and @Jayne have said maybe the thing to do would be to make SAT prep courses available to the economically disadvantaged. This raises an interesting question. Would the methods used by the prep courses be considered too unsavory to be supported by a charitable or governmental organization?

In Kentucky, historically, students headed for private colleges or out-of-state took the SAT, in-state students took the ACT. In 10th or 11th grade, students take the PSAT, which determines National Merit scholarships. Students taking the ACT have the option of take the PLAN assessment in 10th grade as well. In order to make SAT prepping effective, you would have to begin teaching the prepping methodologies as early as the second half of freshman year, which would seem to me to undermine the overall effectiveness and purpose of the test. SAT is coming out with a new middle school level testing, which further muddies the waters.

To me, this is really the core of the issue—lack of appropriate education and resources for students who are identified as academically talented in middle school. This is the point where the rubber hits the road, and economic inequalities begin. There are too few openings in gifted ed for students who are identified as academically talented. Many of the programs that are out there have been diluted by “all children are gifted” mentality. While all children have talents, not all talents are academically oriented. But what happens to those students who are not academically able or oriented, but whose parents want them to be? Do we put the stress on younger and younger children to perform? Admittedly, a little more academic pressure across the board would not be a bad thing.

Perhaps part of the equation is an archaic perspective on the purpose of education. Do we educate for employability or for intellectualism? Not everyone is college material. We do little to help students to identify other pathways to success other than going to college. We need a diversely educated population in order to be successful as a society. We need more understanding of how to get from A to B, and clearer paths that indicate that the application to academics opens and closes certain doors. Perhaps if state universities were unilaterally free to students with certain GPAs and ACT scores (I believe Georgia has a version of this for their residents) then test scores and GPAs would rise unilaterally.

@AlfredaPrufrock , Part of the problem is that education has become a commodity, requiring means by which learning and teaching can be quantified and measured. Maybe this is necessary in order to determine access to higher education, but I am not convinced.

There is some interesting evidence that this attitude does not produce the best results. Jo Boaler of Stanford University has conducted research related to mathematics education using project based learning in classes of mixed ability students. She found that students in these classes actually got an appreciation of mathematics and also did well on standardized tests. While educators are open to this approach, parents for the most part are highly resistant.

http://www.amazon.com/Experiencing-School-Mathematics-Traditional-Mathematical/dp/0805840052/ref=sr_1_4?ie=UTF8&s=books&qid=1234705171&sr=1-4

http://www.amazon.com/Whats-Math-Got-Subject-Important/dp/0670019526/ref=sr_1_1?ie=UTF8&s=books&qid=1234705171&sr=1-1

Another interesting book is the experience of a teacher who incorporated challenging brain teasers into a traditional middle school curriculum. Two interesting findings were that it was not always the same students who solved the brain teasers and that there were several students who changed from math haters to math lovers based on the challenging problems.

http://www.amazon.com/Great-Math-Experience-Engaging-Mathematics/dp/141201509X/ref=cm_cr-mr-title

Parents are responsible for decimating good programs and approaches, as are school districts that don’t believe specialized programs should be funded, and classes should be taught to the common denominator, in order to provide “equal access to education.” My daughter went into a gifted ed program in middle school, after having a horrible experience in a high performing traditional private elementary school. On the first day of middle school, she came home bouncing with excitement. “Mom! Guess what! The math teacher gave us this locker problem to solve and kids talked about it ALL DAY LONG!!! Even at lunch, mom! We talked about math problems at lunch.” By 8th grade, the school was required to include all ability students in the gifted team, causing no end of frustration for the original students, because the non-gifted kids either did not perform at the same level, didn’t understand the work, or, in many cases, didn’t care.

Group projects became nightmarish. Finally, the teacher ability grouped kids on projects, and guess what! The quality of the projects equalized, in terms of skills. All groups ended up with presentations, presentation materials, equal group effort, and written papers. All students performed because they were required to, and there wasn’t the “brainy kid” to pass all the work off on.

“Equal access to education” is a funny thing. When you get a 7th grader who scores a 780 on the SAT in math because he’s been helping his older sister with high school mathematics since 3rd grade and taught himself from her textbooks, but the same kid writes on a 4th grade level, what does “equal access to education” look like? Too often it’s pre-algebra for him, with extra “enrichment” as a tutor to kids who are struggling, because he’s below grade level on language skills.

My daughter was in a blended K-3 class in a parochial school where most parents were not college graduates. The teachers used Box It or Bag It math, Incredible Equations, and lots of number line activities, including physical games. This lasted a year because the school couldn’t provide resources or supplies to support the teachers. The teachers left to go to the public school system, for twice the salary. For a year, all students performed well, and developed number sense. But, like most things, it wasn’t sustainable without commitment to infrastructure.

As an early indicator of academic potential, the SAT is doubtless quite effective, because children in middle school will not, in most cases, have done much preparation for it, although of course if they became standard this would change. Disregarding this problem, it is true that students who score well be supported and offered accelerated programs; beyond simply giving them material more suited to their abilities, this helps foster a spirit of academic excellence which is invaluable for continued achievement. But if courses are determined by the SAT, there is the danger that they will remain focused on the SAT, and will concentrate their lesson plans on improving students’ scores on their Junior year SAT or other tests. It is vital that courses promote creativity in all areas, and the growing emphasis on standardized tests is a wonderful way to squash that tendency. As such, while it may be more of a hassle for school administrations to organize, an essay contest for English or history courses, and a test such as the AMC 8 for math, would be much better for identifying students with creative ability and for encouraging them to pursue that ability.

“All God’s children are lost, but only a few can play the piano.”

In Pennsylvania, the standardized tests start even earlier than middle school. I believe they take the ERBs in fifth grade, and again in seventh grade. Where my kids go to school, they have been doing practice tests since fourth grade.

Standardizing assessment for funding purposes is one thing. Ruining education by standardizing assessment is another.

@AlfredaPrufrock , Having mixed ability classes does not mean that individual projects must have students of mixed abilities. In the books that I referenced, the students were able to choose who they worked with and I suspect that there was a tendency for people of roughly equal ability to be working together. What is of interest is that, as you pointed out, all the groups were productive.

The locker problem that you mention is a classic in recreational math. It is included in the third reference that I gave.

The excitement that students feel when given a challenging problem seems to be a universal experience. Way back when I went to college, I did some substitute teaching between the end of the college term and the end of the public school term. I wat not required to do anything more than babysit the classes. To give the students something to do, I gave them some recreational math problems. I was very pleasantly surprised by the enthusiasm that they showed. The contrast so the usual attitude toward math is so extreme that it really is a crying shame that schools have not gotten the message yet.

I agree with you 100%. Both my daughters were the happiest academically in blended age classrooms. It’s very hard on teachers to prep for blended age teaching, but much more rewarding because you actually get to teach. But schools seem to have fought the cries of preferential treatment from parents of non-gifted children at the same time the same parents are also complaining that more work cuts into after school sports practices. Learning to work with individuals who are not on the same page with you is extremely hard. But I wonder if using academic content is the best way to teach those skills. Perhaps that’s best taught through social service or community event projects, where the playing field is more level in terms of skill set and individual contribution.

I personally would like see schools to do away with grade levels through middle school, and have students work on performance mastery levels. If you can exhibit subject matter mastery, you move on to the next level. If you are too far below expected age/performance level, school provided tutoring kicks in. In the case of high school, I think a model where there are three years of general preparatory education, then testing that directs the student towards two years of trade, business, technical or college gen ed, would be far more effective. While most people benefit from higher education, the reality is that a large segment of the population is still acutely oriented towards education for employment. The expectation that students who are behind will eventually “catch up” has been proven false time and time again.

@Alfreda, I certainly agree with you on mastery-level education, but I can tell you why it will not happen in our public schools any time soon.

The current “fad” in public schools is called inclusion. In this instance inclusion means doing away with most advanced or remedial classes (which is about as close to mastery-level as we have gotten) and including all students into grade levels.

The driving force behind the popularity of inclusion is money. By adopting this principle, school districts have a rationale for disbanding advanced and remedial classes, not purchasing their special educational materials, and firing those teachers. Those kids are dumped into “regular” classrooms, and the teachers are tasked with inventing daily lesson plans that can reach all of their students. Imaging how much time it takes from the normal classroom hour for a teacher to manage 25 kids, 3 of which are can barely read. What happens is that the remaining 22 kids get shortchanged, and the “advanced” kids in the room are completely bored.

If you are against this fad, you are accused of not being inclusive. Clever hijacking of the term by these “educators”.