«

»

MCAT Question of the Day vs. AAMC MCAT

Every so often we get an email from a user saying “Who writes these questions?  Why can’t you post actual questions from the MCAT like other prep companies?”

Why can’t you post actual MCAT questions?

Because we’re not the AAMC, which owns and maintains strict copyright on their questions.  If you could get official MCAT question for free from us, nobody would purchase practice exams from the AAMC!  Not only that but the AAMC only has 8 full-length practice exams available.  This equates to about 1200 questions; fewer than we have available for free… and we provide a new one every day!

Don’t other prep companies provide real MCAT questions in their materials?

Nope!  None of their published materials including their books contain real MCAT questions.  Only if you purchase their courses will other companies purchase access to official exams on behalf of their students.  However, that’s still a ‘rental’ from the AAMC.  We do the same thing for students in our eCourse.

Nobody besides the AAMC is allowed to publish or distribute official MCAT questions.  If you don’t believe us, read the disclaimers in any MCAT book.  If the author wants to avoid a lawsuit, they have to have a disclaimer for their questions just like ours. If anybody other than the AAMC tries to convince you that they will provide you real MCAT questions, they are either lying or are violating AAMC copyright and you want to steer clear of their content.

Who writes your questions?

We do, of course!  Our questions are written by experienced instructors who have been working with MCAT content for years.  They know the ins and outs of how the AAMC writes their questions.  They know this because not only have they been around MCAT content for a long time, they have been writing questions for a long time as well.  We’ve got the data to back up the quality of our questions.

Sometimes your questions are wrong.

It’s true; no denying it.  Every once in a while we write a question that is unclear or just off.  We do our best to send our questions through a reliable editing process but sometimes mistakes do slip through.  We often correct the mistakes as soon as someone points them out, and we’re committed to ultra-high standards for our questions.  With thousands of people reading our questions every day, we’re usually quick about fixing errors.  Please let us know if you find one!

How close are your questions to the real thing?

Notes/Caveats/Methodology:

  1. For simplicity’s sake, we abbreviate the Question of the Day as QaD.
  2. We don’t publish any verbal questions.  As such we only compare our physical and biological sciences questions.
  3. The AAMC does not release data on the difficulty of individual questions, only aggregate data.  Because of the massive sample sizes of our questions and AAMC questions in terms of respondents, we overlay our individual question responses over their aggregate data to compare them.  The major assumption we make is that the distributions of our question difficulty data and the AAMC’s student performance data are similar.  Our analysis shows this approximation is correct primarily due to the large sample size (an excess of 3 million data points on our end; 8 million on the 2011 MCAT’s end)
  4. Most of our users notice we post few passage-based questions.  The reasons for this are varied but primarily logistical.  Part of the experience of passage-based questions is the time you have to read the passage.  If we showed you the same passage 6 days in a row with 6 different questions, by the 6th day you’d already be an expert in the passage which is not how the MCAT works and would make for bad practice.  It also gets very boring to read the same passage for a week straight.  Our questions are more in line with the discrete questions in terms of their content and style.  However, we do also have longer discrete questions which give more background information than many real discrete questions.  These longer questions are meant to simulate the assimilation and critiquing of information that is more in line with passage-based questions.


The Data:

You may wonder if these differences are statistically significant.  Indeed they are; standard errors are less than 0.2% of the scores.  While our mean biological sciences scores and standard deviations are roughly identical to the 2011 MCAT, you can see our physical sciences questions are considerably more difficult.  This holds true for the raw scores (data not shown) as well as the scaled scores.

What about the distributions?  Do ours match up with the MCAT?  See for yourself.


You can see that the biological sciences distribution is roughly similar – consistent with the similar mean and standard deviation.  The physical sciences data are different, with our distribution skewed to the left, contributing to the lower mean.

We hypothesize the reason for the increased difficulty of our physical sciences questions is their heavier reliance on understanding of specific equations and concepts with which students may not be intimately familiar.  MCAT physical sciences focus more heavily on conceptual understanding based on passage information.  MCAT discrete questions are more equation-based.  Since the MCAT distribution of passage-based vs. discrete questions is heavily weighted towards passage-based questions, this may explain the decreased performance of students on our questions versus the MCAT.

The plain reality seems to be that our questions are approximately as difficult if not more difficult than those found on the MCAT.  Though there are fundamental differences in the delivery of our questions (with few being via passages), there are also other important differences including the time spent on each question and the stress level of students while answering questions.  Since we are unable to control for these variables, comparisons must be taken with a grain of salt.

We also make no claim as to how performance on our QaD correlates to performance on exam day, particularly since we do not have data on students’ actual performance.  If there are any students out there who want to tell us their subscores and how those compared to your performance on our questions, let us know (if you wish to maintain privacy, do this via email).

We’ve got over 3 million data points to play with… what analytics would you like to see?

Permanent link to this article: http://portal.mcatquestion.com/mcat-question-of-the-day-vs-aamc-mcat/

3 comments

  1. Johanne

    Hi

    Are you gonna ever gonna publish verbal materials. I think that would be very helpful.

    Thanks

    Josue

    1. M Prep Team

      Hi Josue,

      Verbal is a topic we would love to provide. The Question of the Day format is intended to give students solid but consistent practice in short periods of time. Since the verbal section is not content-based and passages require long periods of time to complete, the Question of the Day format doesn’t serve as an appropriate mechanism for great practice. Instead, we opted to focus on delivering verbal training in the eCourse where we can take a more intensive approach to the strategies and where we have more time to spend with students.

  2. Komron

    Hey, just wanted to say I really appreciate what you all are doing for students. I believe the QaD is outstanding practice in addition to full length test taking. Doing a random question every day, for 365 days out of the year not only helps you keep your head in the game, but also helps you stay in tune with subject matters you may not be studying day to day in your current classes. Thanks for the service, I recommend it to all the premeds I know.

    Komron

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>


6 × 8 =

 
About | Contact | About the MCAT | Disclaimer | My Account

Lee Simonov Services © 2012