It was a momentous occasion for the Florida Department of Education when it received word from an independent company that the Florida Standards Assessment is a valid standardized test, yet despite the general “good news,” some education groups across the state still aren’t completely satisfied with the result.
The seal of validity provided a sort of vindication in the eyes of the Florida Department of Education, which has suffered harsh criticisms over the past year, much of which revolved around skepticism of standardized tests and a rocky roll-out of the FSA.
On Tuesday, the department announced the results of the summerlong study which was ordered to be completed by Gov. Rick Scott and the state Legislature earlier this year.
The department seemed to take the overall green light as an indicator it can move forward with school grades and teacher evaluations.
But for others, Utah-based Alpine Testing’s recommendations highlight significant problems with the test, particularly regarding its content.
FSA’s test developer, American Institutes for Research, created Florida’s test from a Utah standardized test. The study noted while many test items were confirmed to align with the Florida standards, many were overlooked and not confirmed, thus still remaining Utah-oriented.
While the overall grade was a “pass” in terms of validity, the study also raised serious issues for some because it found many test items didn’t correlate with the Florida standards.
A significant chunk -- 33 percent -- of the state’s 3rd English Language Arts (ELA) content didn’t match with the Florida standards (see page 36 of the report). Twenty percent of the 10th-grade ELA content didn’t measure up, either. The study asked a panelist of veteran teachers to match test content with the "intended standard." Often times, panelists chose different standards than the originally intended ones.
For seventh-grade math, 21 percent of the FSA’s content didn’t match Florida standards.
Alpine recommended Florida phase out Utah-aligned test items.
The usage of other state’s content quickly became a focal point of debate among educators and education groups statewide.
Alberto Carvalho, superintendent of Florida’s largest school district of Miami-Dade County, said the existence of Utah-based content could mean the scores were problematic for calculating school grades and teacher evaluations.
“Is it appropriate to use an exam designed for Utah standards for Florida’s students recognizing that these potentially flawed scores would influence not only teacher pay, performance pay, teacher evaluations, and the perception of a letter grade that may not necessarily reflect what teachers taught and what kids learned?” he asked.
The state’s largest teachers’ union, too, was seriously concerned over the department’s decision to move forward and use this year’s FSA scores for evaluations.
“The analysis of the test was conducted in less than 90 days and the people conducting the review have had longstanding ties with the company that administers the test,” the FEA wrote. “But despite the finding that the test was valid, the report by Alpine Testing Solutions raised many of the concerns we have long had with the test: The Florida Standards Assessment tests that students took this spring should not be used as the sole factor in deciding whether students graduate, take remedial courses or repeat a grade, the new validity study said. But that's the way the state plans to use the results, the DOE said.”
In a call with the press, Commissioner of Education Pam Stewart seemed to focus on the big picture, but also indicated staffers had taken a look at literally every question on the FSA to make sure they were all aligned to Florida’s own education standards.
Even the department told Sunshine State News DOE staff had reviewed all items to “ensure appropriateness for Florida students” before they were included on FSA.
But the study itself said many of these items were not confirmed to align with the Florida standards.
The report read:
“While alignment to Florida standards was confirmed for the majority of items reviewed via the item review study, many were not confirmed, usually because these items focused on slightly different content within the same anchor standards.
”The department later admitted Utah-aligned content was left on the test.
The Florida Association of School Superintendents trashed the study’s findings, saying the alignment to Florida standards was too low.
"The report confirmed that the FSA failed to provide close to 100% match with questions,” wrote the group. “The table on page 36 clearly shows that in ELA Grade 3 and Grade 10 – both of which have retention/promotion consequences – only 65% of the items matched Florida Standards. In any gradebook in Florida, 65% = D."
Reach Tampa-based reporter Allison Nielsen by email at allison@sunshinestatenews.com or follow her on Twitter: @AllisonNielsen
