On March 31, 2014, members of More Than A Score met with Chief Accountability Officer John Barker and other CPS administrators to discuss assessment-related policies. This was the third such meeting that has taken place. (The other two were December 12, 2013 and February 10, 2014.)

MTAS participants were Chris Ball, Jennie Biggs, Cassie Creswell, Julie Fain, Wendy Katten and Julie Woestehoff.   CPS participants were John Barker, Didi Swartz (Office of Assessment), Annette Gurley (Chief Officer of Teaching and Learning), Lachlan Tidmarsh (Chief Information Officer), Kati Ellis (Office of Access and Enrollment) and Gavin Doughty (Information and Technology Services).

MTAS’s goal for the meeting was to get clarifications or commitments on a set of very specific questions on policy issues. By far, the most common response we received was that CPS officials were “meeting soon to discuss that and we will get back to you.”

Quick summary:

• Office of Assessment has no internal deadline for the creation of an assessment calendar for 2014-2015. [Response from Barker 4/3/14: The Academic Team will meet during the week of April 7, 2014, to further review the district’s assessment plan for 2014-15.  Decisions regarding the assessment calendar will be discussed at that time.]
• Neither Barker nor Gurley (Chief of Teaching and Learning) knew the origin of the arbitrary percentile cut-offs that are used in the promotion policy.
• Barker said he could not commit to supporting a policy where each school would be required to release a testing schedule at the beginning of the school year to inform parents how much and when standardized testing was taking place at their children’s school. He did agreed to work with MTAS on this and will raise issue with Office of Network Supports.
• Barker did not have any information on a list of incidents of intimidation and mistreatment of opt-out families and students submitted to him and CEO Barbara Byrd-Bennett on March 10, 2014. He offered to bring representatives from Law Department to our next meeting and would get back to us by the end of the week with an update on the concerns we raised. [Response from Barker 4/3/14: The matters raised in the memo of March 10, 2014, are the subject of an ongoing investigation and cannot be commented on further at this time.]
• There is currently no policy in place to systematically and fairly deal with problems that occur during the NWEA MAP administration with respect to how those problems will affect the use of MAP scores for student promotion, SE enrollment eligibility and admission, teacher evaluation, principal evaluation and school quality rating. The MAP test begins on April 28th, and currently the plan is to deal with such issues on a case-by-case basis.
• The Board of Ed passed the new promotion policy at the October 2013 meeting, but there is no official written description of how parts of it are to be interpreted, including clauses that have been present in the policy since October 2009. Gurley did say that she will be talking to the Policy and Procedure department and the Law department this week to provide this.
• There are “conversations happening right now” about the possibility of standardizing the curriculum across all schools in the district as a policy solution for the issue of high student mobility. [Response from Barker 4/3/14: While we did speak about student mobility, I specifically clarified that the discussion of about curriculum was NOT about “standardizing the curriculum across all schools”.  Your summary in the details below more accurately reflects my comments about having a set of common expectations and standards across schools: “Barker said the Academic Team does talk about having ‘commonality’ in the district, and they will continue to talk about ‘standardization.’ They think such standardization would benefit students who change schools within the district and schools with high mobility rates.” I would add here that the Common Core State Standards provide the framework around which these discussions are taking place.]

Full details on these discussions below. In bold, you will find MTAS’ request or question. Beneath that you will find our notes of the CPS response and our follow-up questions and comments.

A detailed written response to the MTAS alternative promotion policy which we presented at the December 2013 BOE meeting. The response should include which specific elements of the proposal are not currently being considered for inclusion in district policy, which elements are under consideration, and a timeline and process for revision of the policy which should include parent, teacher, student and community input.

Annette Gurley said that the MTAS Alternative Promotion Policy will not be used. CPS will move forward with their current plan—the Promotion Policy approved in October.  (The new policy is identical to the previous promotion policy that has hurt thousands of students over the last nearly two decades but it uses percentiles from the NWEA MAP instead of the SAT-10 section of the ISAT.)

PURE has filed a FOIA, asking CPS for studies to support CPS’s use of NWEA/MAP in high-stakes decisions such as the Promotion Policy. CPS has asked for an extension.

A written process in the CPS promotion policy that specifies how a student may be promoted to the next grade (without summer school attendance) in the absence of NWEA scores. This should include a clearer, more detailed, enforceable parent appeal process.  

Here’s some background: the Board of Ed passed the revised promotion policy in October but there is no official written description of how this paragraph is to be interpreted:

“The District-Wide Assessment (DWA): The district-wide assessment will be the Northwest Evaluation Association’s (NWEA) Measures of Academic Progress (using the highest score from the last two annual assessments) or other norm-referenced assessment administered by the district. Students with no DWA scores in either reading or math will be considered achievement level 2B. If a student’s DWA results are incomplete or inaccessible through no fault of the student, the Office of Teaching & Learning shall make a promotion determination using the best available data consistent with the standards described in the Elementary School Promotion Policy.”

Barker explained that CPS has talked about it and that there is no resolution on the students in promotion years who may not have a NWEA/MAP score.

MTAS pressed that CPS needed a clear written interpretation of how the policy applies to these students. The MAP testing window is opening soon (April 28) and parents need to know.

Gurley said that she will be talking to the Policy and Procedure department and the Law department this week to provide this for us.

(Note that a very similar sentence has been part of the promotion policy (Sec II.C.1.) since at least October 2009:  “If a student’s DWA results are incomplete or inaccessible through no fault of the student, the Office of P – 12  Management shall make a promotion determination using the best available data consistent with the standards described in this Policy.”)

A written policy that each school must provide all parents by the first week of the 2014-2015 academic year a comprehensive schedule of all standardized tests (federal, state, district, network, and school-level ) that are administered at each school, excluding any tests created by teachers exclusively for classroom use.

MTAS volunteered to help define what type of test would be included in a school’s individual assessment calendar.

Barker would not commit to this. He said it was a “Network issue” (meaning it falls under the purview of the Office of Network Support), but that he was willing to have a conversation with the Network Chiefs to work toward how this would work. Barker also said FACE could be used to educate parents on these tests. MTAS said we can do that too. Barker said he will have feedback on this at our next meeting, and he will invite the Network Chiefs to our next meeting.

There was also an unanswered question on the table about the authority of networks to mandate certain tests that are not district-mandated.  Barker also said in response, “We’re having conversations about that” too.

A response to our list of incidents of student and parent mistreatment related to ISAT opt out.

CPS is “still investigating” our complaints about student and parent mistreatment during ISAT. (We provided a list of incidents on March 10th.) There were “opportunities for improvement” in how the whole thing was handled, so MTAS asked for formal guidance on the treatment of parents and students who opt out of tests. We want a clear policy so principals have guidance on how to treat opt-out families ethically. Families at 90 schools opted out.

There was a discussion about how the punitive culture of CPS is as much to blame as no clear policy. A principal’s reaction to the opt-outs is a product of the system.

Barker spoke of inviting representatives from the Law Department to talk to us about the investigation at the next meeting. He is part of a meeting with them this week. We stressed that we wanted clear guidance for principals. This is the first time that a mass opt out took place. CPS should not let the mistreatment of families and students that took place happen again. Barker said he would have a response to this request by the next meeting with an email update by the end of the week.

How were the percentile ranks used in the promotion and summer school policy set? To the best our knowledge, there is no written research or analysis on the suitability or validity of these percentiles for promotion purposes. If there is any, could you please provide us with a copy?

Neither Barker nor Gurley knew the origin of the arbitrary percentile cut-offs that were used in the promotion policy for the ISAT.  These percentiles have now been translated to the NWEA MAP so that they will result in about the same number of students being promoted.  (See Appendix A in this document for a timeline of the history of high-stakes testing in CPS.

A written statement on how technical failures that interrupt a student’s MAP test or fail to save student responses will be handled. This should include how  such failures will be factored into the implementation of SQRP, SE eligibility and admission, student promotion, and teacher and principal evaluation.

We were joined by Lachlan Tidmarsh and Gavin Doughty (in charge of PARCC readiness).

MTAS explained the following known situations while taking the NWEA/MAP: crashing computers; couldn’t get back to the part of test where it crashed or stopped; completed test but no score issued followed by test retake then student had 2 scores the next day; etc.

What can be done? Report incidents? How? What should a teacher do? The testing window for NWEA MAP begins in less than four weeks. Some schools cannot be on internet during the entire testing window of NWEA MAP due to bandwidth issues.

Tidmarsh’s response: NWEA had application issues in the beginning. He has two children in CPS so he takes this very seriously. They’ve received feedback from teachers and have worked with NWEA to resolve and to improve the application. During Winter testing, they had a “clear run.” They monitored bandwidth use during the Winter testing session. They have a checklist they send to schools before the testing window opens so the school is ready and start up issues can be resolved beforehand. CPS has a Help Desk with an “expedited path” directly to NWEA. It had 250 tickets at mid-year; three were bandwidth issues and were resolved. Some issues were delayed start time of the test. They are ready for the EOY tests.

More from Tidmarsh: Having proper bandwidth is an issue of getting fiber into all schools. CPS is 120 schools short of having fiber in all. By September 30, 2014, all schools will have fiber. [Response from Tidmarsh 4/6/2014: “[W]e are actually only 70 schools away from have fiber District-wide by 9/30/14, not 120 as I mentioned in the meeting. I’ve been using the 120 number for a while, not taking into account that these installations are ongoing. The correct number is 70 and we remain on track for September.] They did communicate to schools to not use internet for other non-testing purposes during testing. Can be nuanced on messaging due to the amount of fiber per building.

120 schools don’t have the wireless network to support the testing; this is a matter of access points, not bandwidth. [Response from Tidmarsh 4/6/2014: “120 schools have wireless networks that have not been upgraded in 5-8 years. Planning for these upgrades is in progress and we anticipate completing the upgrades before October of 2015. These schools, if not using hard-wired labs, are most at risk of testing related performance problems, and as such are being carefully monitored by the District technology team.”]

Suggestions from Tidmarsh on what to do if you encounter problems during NWEA testing: Document the details. Call the Help Desk.

MTAS did ask if CPS can get a detailed list from NWEA on the reported problems (tickets) from the Winter testing so they can determine what some of the potential problems can be for EOY testing. Didi Swartz said she will look into that.

More from Swartz: There will be a comprehensive NWEA proctor training via a webinar. MTAS asked if we could see it? The video will be housed internally so she will look into how to share it with us, and this training video should be live Thursday, April 3rd or Friday, April 4th. CPS has been in contact with Charlotte—another large district using NWEA, and they report “few incidents” in their most recent testing window.

MTAS pressed and pressed some more for a clear policy for schools to follow when a tech error occurs. MTAS also brought up that maybe standardized test administration on computers is not the best option if many high stakes decisions are pinned to it when there is such room for error and problems.

The bottom line: There is currently no policy in place to systematically and fairly deal with problems that occur during the NWEA MAP administration with respect to how those problems will affect student promotion, SE enrollment eligibility and admission, teacher evaluation, principal evaluation and school quality rating. The MAP test begins on April 28, and currently the plan is to deal with issues on a case-by-case basis.

Will the Common Core Quarterly Benchmark tests be required for 2014-2015? If so, why will these be required when the PARCC and the NWEA MAP will both be in use?

The “Academic Team” is meeting this week to discuss. Barker said he could email us this week.

They are not yet sure if the benchmarks will be required in next year’s assessment calendar. Swartz said they are hearing from a lot of teachers that the CIM entry is cumbersome. They have changed (or will change?) the data entry process to be less cumbersome. (We don’t have any details or access to that internal system.) Feedback from teachers is critical on this since there is some sense that teachers are giving them useful information on how the benchmarks are working/not working.

(Since parents don’t even know these tests are happening, it’s teachers who will have the most influence there.)

Target Date for an Assessment Calendar for 2014-15?

Barker said the Office of Assessment has no internal deadline for the creation of an assessment calendar for 2014-2015. It will be “soon.”

We also ask for copies of the Office of Assessment’s policy on weekly K-2 sight word assessments at Level 3 schools implemented by Office of Network Support and clarification of the policy on the mandated literacy test and the MAP test for 2nd graders.

This is an Office of Network Support initiative. Barker will ask the Office of Network Supports about this.

Then there was a discussion about “rumors of the standardized curriculum” being mandated from the top down, citywide. MTAS asked about the current district-wide moratorium in purchasing new curriculum materials and whether the moratorium was a precursor to everyone having to buy the same curriculum at the same time.

Barker said the Academic Team does talk about having “commonality” in the district, and they will continue to talk about “standardization.” They think such standardization would benefit students who change schools within the district and schools with high mobility rates.

MTAS challenged them on the benefits of such standardization.  Swartz explained that two years ago a moratorium was placed because Common Core was so new, and CPS wanted to wait for materials to catch up and be truly Common Core aligned. Swartz said with respect to standardization that there “needs to be some allowance for the work people have done that is high quality.”

Finally, in 2nd grade, if the school has chosen MPG as its K-2 literacy test, 2nd graders should take the MPG in fall, the MAP in winter and spring.  If the school has chosen TRC or another standardized literacy test, second graders will likely take TRC (or the test of choice) three times, and then also take the MAP in winter and spring.  No school should be giving K-2 both the TRC and the MPG, however. MTAS reported that at least one school was giving both. Swartz was going to look into that.