Home    Contact    Customer Service    Webmail
1st choice   1st Choice   academics   academies of nashville   academies of Nashville   Academies of Nashville   academy   academy at hickory hollow   academy at opry mills   accountability   achievement   achievements   act   ACT   administration   adult education   advanced academics   advanced placement   Adventure Science Center   after school   afterschool programs   agriculture   alumni   alumni news   anna shepherd   Annenberg   annenberg institute   announcements   annual report   antioch   Antioch   AP   apollo   art   arts   assessment   assessments   astronaut   athletics   atlanta   audio   AVID   award   awards   ayp   back to school   Back to School   bailey   Bailey   band   baseball   basketball   Baxter ALC   bellshire   benefit   big picture   Big Picture   BizTown   blue cross blue shield bowl   board   Board   board meeting   board of education   Board of Education   book 'em   bowling   breakfast   brick church   bridge school   budget   bully   bus driver   business partners   cafeteria   cafeterias   calendar   call to action   Cameron   cane ridge   capital funds   capital projects   career   career fair   careers   celebration   celebrations   celebrity sightings   ceo champions   chadwell   Chadwell   Chadwell Elementary   chamber of commerce   charlotte park   charter   charter schools   charters   cheekwood   cheerleading   chess   chinese   class schedules   clubs   CMA   cmt   coach   cohn   cole   college   college board   comcast   comcast cares day   common core   common core standards   communication   communications   communities   community   community gardens   community schools   community services   competition   competitions   conexion americas   conference   confucius institute   contact   contest   contests   continuity   coordinated school health   cora howe   counselors   country music association   country music hall of fame   croft   Croft   cross country   culinary   culinary arts   Customer Service   Dalewood   dan mills   Dan Mills   data   debate   department of education   director   diversity   donate   donation   donations   dr. jesse register   dr. register   Dr. Register   Drake   dupont hadley   eakin   Eakin   early childhood education   earth day   East End Preparatory   East Literature   East Nashville   education   EL   elementaries   elementary   elementary school   elementary schools   ELL   email   encore   Encore   end of course   end of course exams   end of the year   engagement   engineering   Engineering   English   english classes   english learners   English Learners   enrollment   entertainment   ESPN   essay   essay contest   evaluation   event   events   exams   exceptional education   Exceptional education   exercise   expansions   facebook   Facebook   facilities   fall application   fall break   families   family resource center   family resource centers   female   financial aid   financial literacy   fine arts   firefighters   first day   first day of school   fitness   flu   food   food services   football   ford   ford motor company fund   Ford NGL   Ford NGLC   foreign language   fund raiser   fund raising   funding   fundraiser   garden   gardens   gateway   Gateway   ged   gifted services   glencliff   Glencliff   glendale   glengarry   glory days   gnash   golf   goodlettsville   governor's school   gower   Gracie Porter   grades   gradespeed   grading   graduate   graduation   gra-mar   grant   grants   hall of fame   hands on nashville   harris-hillman   hattie cotton   Hattie Cotton   head   Head   health   hickory hollow   high school   high schools   High schools   hillsboro   hillwood   hispanic   hispanic heritage month   history   holidays   HON   honors   hull-jackson   human resources   hume award   hume-fogg   Hume-Fogg   hunters lane   ib   IB   iLeader   immunizations   INA   Inc.   inclusion   inglewood   innovation   innovation cluster   innovation school zone   Innovation Zone   international   international baccalaureate   international studies   internet   internship   internships   intersession   iTeacher   it's the little things   izone   iZone   JA   jobs   john early   johnny majors   JROTC   julia green   junior achievement   kecia ray   keep the music playing   kennedy   kindergarten   kirkpatrick   kiwanis   lakeview   language arts   latino   Latino   leaders   leadership   Leland Melvin   libraries   library   library on congress   limitless libraries   literacy   literature   little league world series   litton   Litton   live-blog   lorenzo butler   madison   magnet   Magnet   magnet schools   magnets   maintenance   make-up   maplewood   Maplewood   maplewood high school   maps   mariachi   mark north   Mark North   Martin Luther King Jr. Day   mascots   math   Math   Mayor   mayor dean   mayor karl dean   Mayor Karl Dean   mcgavock   McGavock   mckissack   Mckissack   mcmurray   media   meeting   meigs   mentor   Menus   Meritorious Budget Award   metro   metro council   metro government   metro health department   metro nashville   metro parks   Metro Schools   middle   middle college   middle school   middle schools   mlk   MLK   mnps   mnps achieves   MNPS Achieves   montessori   moving   mt. view   mtsu   Murrell   museum   museum magnet   museums   music   music city music council   music education   music makes us   NAPE   napier   NASA   nashville   Nashville Big Picture   nashville interscholastic sports league   nashville predators   nashville prep   nashville public education foundation   nashville public library   nashville teaching fellows   nashville tools for schools   national   national football foundation   national honors   national media   NAZA   nclb   NCLB   nea   neely's bend   neighborhood schools   news   NFL   nicole cobb   non-traditional   now hiring   NSA   nurtition services   nutrition   nutrition services   Nutrition Services   Oasis   office of innovation   old center   oliver   olympics   one nashville   ONE Nashville   open house   optional schools   options   outdoor classroom   overton   PAC   paideia   Paidiea   parent classes   parent connection   parent event   parent events   parent involvement   parent invovlement   parent university   parental engagement   parental involvement   parents   parks   parthenon   partners   pearl cohn   pearl-cohn   Pearl-Cohn   pencil   Pencil   pencil box   pencil foundation   PENCIL foundation   PENCIL partners   percy priest   performance   performances   performing arts   photos   physical education   physical fitness   poetry   police   prek   principal   principal of the year   principals   professional development   professionals   programs   pta   public meetings   public notice   publications   race to the top   Race to the Top   radio   raymond berry   reading   recognition   recognitions   recruiting   referee   reform   renovations   report   report card   report cards   reports   research   resources   reunion   Reward Schools   robert churchwell   robots   rose park   Rose Park   RTTT   ruby major   safe schools   safety   sage advice   SAT   scholar athletes   scholarship   scholarships   school choice   school choices   school counseling   school for science and math at vanderbilt   school meals   school options   school supplies   schools   science   Science   SCORE   sec   seniors   service   services   shwab   signing day   simon youth foundation   smithson-craighead   snow day   snow days   snow patrol   soccer   social media   social studies   softball   spanish   special events   specialty schools   speech   spelling   spelling bee   sports   sports hall of fame   Sports hall of fame   Sports Hall of Fame   ssa   ssmv   SSMV   staff   Stand for Children   standardized tests   standards   stanford   start of school   state championship   state department of education   stem   STEM   stratford   Stratford   student achievement   student assignment   student records   student services   students   success   summer   summer camp   summer camps   summer programs   supplies   surprise!   tax   taxes   tcap   TCAP   teacher of the year   teacher-led   teachers   teaching   technology   Technology   television   tennessee titans   tennis   test scores   testing   tests   textbooks   the academies of nashville   The Academies of Nashville   the academy at hickory hollow   the academy at old cockrill   the academy at opry mills   The FANS   the hermitage   the north sports report   tips   tndoe   tojo yamamoto   tpac   track   track and field   transformation   transition   transportation   travel   trivia   TSSAA   TSU   tutoring   tvaas   united way   university of tennessee   update   update your info   us department of education   USDA   vaccinations   vanderbilt   Vanderbilt   vanderbilt university   video   virtual   virtual school   visual arts   volleyball   volunteer   volunteers   weather   websites   wes elrod   west end   whites creek   Whites Creek   workshops   wrestling   wright   writing   youth safety   zoning  

Researchers condemn shoddy methods, analysis of test data investigation

UPDATE (3/27/12): The Atlanta Journal-Constitution has removed any references to Two Rivers Middle School from the original story.

UPDATE (3/26/12): The Washington Post agrees with our objections

The Atlanta Journal Constitution (AJC) has hired a marketing firm to tout its investigative story on testing data from several large school systems despite being informed of numerous flaws in its methodology and analysis. The story is scheduled to appear in this Sunday’s edition and, while Metro Schools has not seen the story, the paper shared its data and methodology earlier this week in its first contact with the district. The information provided is questionable and shows a complete lack of knowledge or understanding of the system and students. Other school systems contacted have voiced similar protests of the data.

“We looked at the data provided, along with the methodology for their analysis, and immediately identified numerous flaws, as did an independent statistician from Vanderbilt,” said Dr. Paul Changas, executive director for Research, Assessment and Evaluation for Metro Schools. Dr. Changas was the first statistician hired by the State of Tennessee to assess student achievement results.

The AJC has based its analysis of school testing data from the nation’s largest school districts by “flagging” any movement in student achievement that is extremely high or low. Metro Schools’ has provided detailed information to the AJC outlining the serious flaws in the information provided to the district and in the methods they used to calculate their data. Some of those include:
  • The AJC analysis assumes that students in one grade level at a school one year are the same students that were there the previous year in the previous grade.
    • Metro Schools has mobility rates between 35%-40%.
    • Zoning changes have impacted student enrollment.
    • A significant number of ALCs (Alternative Learning Centers) and special schools (including homebound) are flagged in the data; these schools often have very fluid populations - ALC populations have 100% annual turnover.
  • This AJC information flags schools with unusually high numbers of discrepancies from predicted scores – both high and low.
    • 77% of the cases flagged in MNPS were due to low performance (significantly below predictions).
    • The negative scores we saw were not part of a pattern of high gain one year followed by significant loss the next (as would be expected if there were improper testing procedures).
    • There was not a single case of unusually high gains flagged for 2011.
  • The AJC data we received had obvious errors, including children who were absent from testing being assigned a “zero” rather than being excluded from the analysis - resulting in average scores that were below the minimum score possible.
  • Tennessee adopted much more difficult academic standards in 2010, resulting in a significant decrease in student test scores.
  • Metro Schools has the highest population of English Learners in the state, with approximately 1 in 4 of our students coming from non-English Language Backgrounds. National data show EL students typically make among the largest gains from year to year as they learn the English language; several of the schools flagged for positive gains have high EL populations.
  • The Tennessee Value Added Assessment System (TVAAS) uses much more sophisticated analytical techniques (Henderson’s mixed model methodology) and follows actual cohorts of students to analyze gains of teachers, schools, and districts and TVAAS results have typically been comparable to statewide trends for several years.

Read a detailed list of issues with AJC's methods

Other information provided to the AJC included information from Dr. Dale Ballou, associate professor of Public Policy and Education with Vanderbilt University’s Peabody College of Education, who has reviewed Metro Schools’ testing results for a number of years. Dr. Ballou has reviewed the data provided and found some statistically impossible anomalies. In addition, Dr. Ballou has been involved with an independent analysis of test gains of MNPS middle school math teachers over three years (2006-07 through 2008-09). This review, conducted by Dr. Brian Jacob of the University of Chicago on behalf of Vanderbilt University, did not show any unusual or suspicious patterns.
  • The analysis was conducted as part of the Project on Incentives in Teaching (POINT) study, to determine whether tying significant financial incentives for teachers (up to $15,000 per teacher per year) to student achievement made an impact.
  • The analysis conducted by Dr. Jacob was specifically to determine if evidence existed of invalid test scores. None was found.
“We are shocked and disappointed the Atlanta Journal Constitution has included our district in their investigative report to be released Sunday, despite these and a number of additional flaws in their analysis brought to their attention prior to its publication,” said Director of Schools Dr. Jesse Register. “Their lack of understanding of our schools and district is reflected in the data we have seen.”

Michael Casserly, executive director for the Council of the Great City Schools, a national organization representing large school districts, has also criticized the Atlanta Journal-Constitution’s approach, saying, “These are serious charges that should be supported by the strongest evidence. That strong evidence has not been provided.”

Read the full statement from the Council of Great City Schools

Tags:
ADA Symbol  ADA Compliance 2601 Bransford Ave. Nashville, TN 37204 615-259-INFO (4636) Home | Sitemap | Terms of Use