Posts

Ongoing Performance Monitoring to Drive Gains in Student Achievement

By Annice Weinstein, Senior Manager, Assessment Data & Analysis

November 2020 

EdTec’s charter school data and assessment experts support school leaders in developing plans to monitor progress on identified measurable outcomes, and schools that follow a defined process have shown considerable gains in student performance.  

Amid all the uncertainty and unexpected changes this school year, it is important not to lose sight of your charter goals for academic achievement and student success, particularly given the current political climate. With some forward planning, you can keep your school on target by defining your measurable outcomes, monitoring progress, and regularly sharing results with your school community throughout the year to reinforce common goals.

Identify Key Measurable Outcomes 

The first step is to identify the primary measurable outcomes for your school. This should include key statewide accountability measures such as state tests, English learner progress, chronic absenteeism, suspension rates, and graduation rates. For academic achievement, consider including measures that can be evaluated throughout the year as well, such as performance on common benchmarks or normed assessments, or measures of course performance like the percentage of students with passing grades in all core courses. Make sure to include measures that are representative of the unique qualities of your charter. For example, set college application/acceptance rates or post-secondary outcomes as a measure for a college preparatory charter, or student survey results if social-emotional growth and a positive school climate are a focus. And remember to engage your stakeholders – teachers, administrators, parents, students – to determine the best measures of success for your students.

Edtec helps charters prepare their student outcome data for presentation at stakeholder engagement meetings.

Create a Cycle of Data Inquiry 

Once your team has identified the key measures for your school, start to map out your calendar of when the data will be available and your schedule of reviewing progress with your stakeholders. The level of analysis will vary depending on the group but do your best to present the same measures to all stakeholder groups. For example, your teachers may take a deep dive into NWEA MAP performance, overall and by strand, for their whole class, groups of students and individual students. For your parents, it may be sufficient to let them know the MAP subjects tested and how students schoolwide are progressing towards an annual target. The frequency of review will also depend on the data type. Chronic absenteeism could be monitored monthly, grades by semester, benchmark tests after each administration, and graduation rates annually. Determine what measures make sense to share at different points throughout the year.

Define the Data Collection Process 

The most complicated part of the process will be determining how you will collect data on key performance measures in a way that allows for a streamlined analysis. For some measures, it’s a matter of tasking a staff member to extract data from your existing systems on a regular basis, such as downloading a state test data file from its reporting system or aggregating attendance data from your student information system (SIS). For custom measures, it helps to house the data in a single online reporting system, such as Otus or Illuminate DnA. If you plan to collect information from each individual teacher, establish templates for data collection to make it easier to aggregate scores across classes.

Using an assessment and learning management system such as Otus can help to streamline the data collection and reporting process.

Plan for Streamlined Reporting 

Finally, think about how you plan to report the information. For example, if you want to know how many of your students are reading at or above grade level by the end of the year, do not just collect students’ raw scores on their reading assessments. Include a field in your template that translates the raw score to a proficiency score of Below Grade Level, Approaching Grade Level, or At or Above Grade Level, so you can easily see how many students are At or Above Grade Level. If you select NWEA MAP performance, it makes more sense to track progress towards a student’s growth target than it does to track whether the student is on grade level, since the system reports focus more on growth than grade level equivalents.

No matter what measures you select, make sure you can also disaggregate the data by significant student subgroups. This is easier to do if you store the data in a system such as Otus or Illuminate DnA, but this analysis can be done manually as well as long as all the data sources include a unique student identifier, such as the SIS student ID or state student ID. Performance of student subgroups can be a key factor in charter renewal decisions, and many charters excel in supporting historically disadvantaged student groups, so it’s good to stay on top of your progress.


EdTec’s data and assessment experts provide NWEA analysis services, Illuminate DnA implementation and customization support, as well as setup and licensing for the Otus Data, Assessment, and Learning Management solution at a low per-student cost. Additionally, we help charters to plan for stakeholder engagement meetings and to present outcome data to stakeholders, authorizers, and the community. Through EdTec’s new partnership with the National Student Clearinghouse®, we offer the StudentTracker® service to charter schools bundled with our robust, charter-specific data services.

If you’re interested in learning more about how we can support your school in these areas, reach out to us at assessment@edtec.com.

Otus logo and images are the property of Otus, LLC and reproduced with permission. 

California Charters, Know the Impact of Participation Rate on Your School’s Academic Indicators!

By Annice Weinstein, Senior Manager, Assessment Data and Analysis

March 29, 2019

In 2018, the California Department of Education added participation rate as a factor into the calculations for the Academic Indicators on the CA School Dashboard. But how exactly does that affect your school?

For both ELA and Math, the goal is to have a participation rate of 95% or higher on the Smarter Balanced Summative Assessments (SBAC) and the California Alternate Assessments (CAAs.) If the participation rate falls below 95%, a fourth of a point (.25) is deducted from the Distance from Standard (DFS) for that subject for each percentage point below the target 95%.

For example, if a school had a participation rate of 91% on the SBAC/CAA ELA and a DFS in ELA of 9:

95% – 91% = 4 (points below the target participation rate)

4 x .25 = 1 (deduction amount from the DFS)

9 – 1 = Adjusted DFS of 8

Participation rate is calculated for every subgroup with 11 or more students and factored into the Academic Indicators for all reported Dashboard subgroups.

The following students are not included in the participation rate calculation: students absent from testing due to a significant medical emergency who are also flagged with the medical emergency condition code on the CAASPP file, and for the ELA portion of the SBAC/CAA, English learners enrolled in a U.S. school for less than one year. Parent waivers do not exempt students from the participation rate calculations.

Schools are also responsible for testing students who transfer in during the beginning 14-day grade period or during the Accountability Testing Window. These students will be included in the participation rate calculation. Schools are not responsible for testing students who transfer in during the ending 14-day grace period or transfer out during the beginning 14-day grace period. However, schools are responsible for testing students who transfer out during the Accountability Testing Window or during the ending 14-day grace period.

To see how participation rate may have affected your school’s Academic Indicators in 2018, schoolwide and for each significant subgroup, enter your school name into the box on this site:

https://public.tableau.com/profile/aweinstein#!/vizhome/2018_Participation_Rate_Impact/ParticipationRateDashboard

CALPADS Update for CA Charter Schools

By Gerald Cockrell, Data and Categorical Program Specialist

September 19, 2018

For California charter schools, it can be hard to stay on top of all the changes to CALPADS. Use the list below as an easy checklist to make sure your school isn’t forgetting anything this year. Check out our previous CALPADS post to see our list of the most important CALPADS reports to prepare for Fall 1.

Important Changes to CALPADS for the 18/19 School Year

  • There are new CALPADS submission certification deadlines for ‘18-‘19 and they are much earlier than previous years. Please plan accordingly, especially for the Fall 1 submission which now ends around Thanksgiving instead of the end of January.
    • Fall 1 Certification Deadline: 11/21/18
    • Fall 1 Amendment Window Deadline: 12/7/18
    • Fall 2 Certification Deadline: 2/1/19 (No Amendment Window)
    • EOY 1 – EOY 3 Certification Deadline: 8/17/19 (No Amendment Window)
  • In addition to the shorter submission windows CALPADS also has a significant outage planned for 9/21-10/01 to make major system changes. Please plan accordingly. It’s also important to get any SSID’s or other information from CALPADS you may need for assessment or ELPAC testing.
  • New students with a non-English primary language must have an ‘18-‘19 enrollment record and a “TBD” SELA record in CALPADS in order for them to show up in TOMS and be tested with the Initial ELPAC.
  • This update is not new, but it is important enough to warrant an annual reminder: all National School Lunch Program (NSLP) Free and Reduced Price Meal (FRPM) applications for the year MUST BE RECEIVED BY 10/31/18 for the student eligibility to count towards your official FRPM or LCFF Unduplicated counts.

To help you prepare for Fall 1 and make sure your charter school’s data is accurate for funding and accountability, check out our previous CALPADS post about the most important CALPADS reports.

We Need Proof: Capture & Demonstrate Student Performance across a Data Spectrum

by Annice Weinstein, Senior Manager, Assessment Data and Analysis 

May 23, 2018

With so much going on at your charter school on a day-to-day basis, it can be difficult to remember what data your staff needs to track throughout the year. This data is necessary to stay on top of accountability requirements and be able to demonstrate student performance to your stakeholders, as well as for your LCAP, charter renewal, WASC, and grants. This article takes a step-by-step approach to help you evaluate your data needs and establish processes for collecting that data.

Start with your Charter

When your school’s founding team wrote the charter, they laid the foundation for what the school aims to accomplish, student performance goals, and how it plans to measure its progress over the duration of the charter period. Because your LCAP is an annual reflection of the school’s performance and plan for improvement, aligning the goals and measurable pupil outcomes in your LCAP to your charter is critical. This will also save you time when you renew your charter, as aligning the two documents is a requirement for renewal.

As a next step, you’ll need to track progress towards the measurable outcomes you set in your LCAP. For example, if one of your student performance goals is that all students enrolled at your school since ninth grade will graduate and be accepted to college, then you need to have processes in place to track progress towards graduation requirements and college acceptance rates. To accomplish this, you could assign a member of your data team to collaborate with the school’s guidance counselors and college counselors to determine the best way to gather this information. If one of your metrics for academic achievement is that all students enrolled at your school since kindergarten will be reading on grade level by the end of 3rd grade, then you need to have processes in place to track reading levels for a cohort of students over time.

Data Integrity

You won’t be able to evaluate your progress and student performance toward LCAP goals and tell your school’s story without consistent, reliable data. What’s more, your CALPADS data will be used to calculate the CA School Dashboard state indicators, so it’s extremely important that it’s complete and accurate. Review all CALPADS certification reports carefully. Take the time to download all student-level state test results from TOMS, CELDT, ELPAC, and testing systems to a safe place at your school. The data may not always be available to download when you need it; for example, TOMS only houses two historical years of data.

For all local test data, make sure to include student IDs so you can calculate longitudinal progress. Include proficiency levels where applicable to make it easy to determine the percentage of students on grade level or meeting the standard. Make sure the team in charge of data at your school is aware of the important role they play in compliance requirements and storytelling, and work with them to establish standard processes for tracking and reporting all data. You’ll want to make sure that everyone involved in these processes is kept informed of critical updates and timelines.

Use Data to Tell Your Story

CA School Dashboard Indicators

When it comes time to tell your story, it’s important to present the school’s performance in a clear, honest, effective way. In addition to the good news, you’ll also need to communicate dips in performance and indications of achievement gaps. You can use your Dashboard indicators to identify achievement gaps by identifying any student groups performing two or more levels below the school’s overall performance. You should also be prepared to explain how you are using data to assist in improving performance or closing the achievement gap.

Your Dashboard indicators are also useful for identifying and highlighting progress in student performance, such as an improvement in the test performance of English Learners. Whenever possible, try to identify specific programs or lack of programs that can be tied to progress or setbacks; this will weave a thorough story and set the stage for you to provide related recommendations about the best way to allocate resources going forward.

Local Metrics

You can also use local metrics to highlight your school’s performance. If you’ve had success in a particular area, you’ll want to make sure to share this with your stakeholders. Here are a few examples of metrics that might make sense for your school to track and report:

  • College Application, Acceptance, or College-Going Rate
  • AP Course Enrollment or AP Pass Rate
  • GPA
  • Participation in Programs (extracurricular activities, arts & science programs, leadership)
  • SAT/ACT Scores
  • Fundraising Success
  • Community Service Hours
  • Re-Enrollment Rate or Waitlist Count (to indicate satisfaction with or interest in your school)
  • Results of Parent/Student Surveys

Performance Analysis

There are two main ways to demonstrate student performance when telling your school’s story: longitudinal progress and comparison view. The longitudinal view compares performance for the same set of students over time. The comparison view shows school-wide performance by grade level or subgroup compared to demographically similar, resident, or district schools. When comparing performance, be sure to select comparison schools based on specific criteria such as similar demographics or geographic proximity. In the end, it all goes back to data integrity, as both views depend on access to complete, accurate data.

Summary

The most effective way to prepare to demonstrate student performance at your school is to outline a data collection strategy and plan. Make sure the relevant staff are on the same page regarding your school’s goals and implementation of those goals, so you are capturing the information you need to tell a powerful story. Schedule time throughout the school year to review progress on the measures established in your LCAP, accurately maintain your CALPADS data, and thoroughly understand your Dashboard. All of this will prepare you to explain both the progress and struggles reflected in your data.

Find out more about EdTec’s Student Performance Services.

Get Your School’s Performance Data Renewal Ready!

By Annice Weinstein, Senior Manager, Assessment Data & Analysis

December 14, 2017

Charter renewal is a very time intensive process, and it can be even more demanding if you haven’t already collected the necessary performance data. The following three steps will set you up to have your data ready when the time comes to start working on your renewal petition, so you can focus your energy on putting together a sound analysis and a strong narrative that highlight your school’s achievements!

  • Download your student level data files for state tests as soon as the state makes them available. These files may not be available in your account when it’s time for renewal, so it’s best to download the files annually as they are released. For example, the Test Operations Management System (TOMS) currently stores just two years of summative test results, so make sure to have your LEA CAASPP coordinator download the files as soon as the state indicates they are ready. You can find the files in TOMS (https://caaspp.ets.org/) under Reports (on the left), LEA Reports. Scroll down to the bottom to download the Student Results Report-Student Score Data Extract for each year available. CalTAC has recently removed the 2014-15 files, so you may need to contact them directly if you didn’t already download these files. You may reach CalTAC at 800-955-2954 or caltac@ets.org.
  • Keep track of your comparison schools. Does your authorizer require comparisons to specific schools? Are there other schools in your area, or schools with demographics or programs similar to yours, that you strive to outperform? If most of the relevant comparison schools are within the same district, take a look at the California School Dashboard Five-by-Five grids to see how your school stacks up on the Dashboard indicators: http://www6.cde.ca.gov/californiamodel/. And keep an eye on your Academic Accountability Report with the California Charter Schools Association (CCSA): http://snapshots.ccsa.org/aar. CCSA produces a Similar Students Measure that will give you an idea of how your school’s performance on the SBAC compares to that of demographically similar schools.
  • Select LCAP performance metrics that track longitudinal progress for the same set of students. This will help you stay on track with annual data collection and give you a boost when you want to share your longitudinal progress on local assessments with your authorizers. The metrics can include performance on NWEA MAP, developmental reading assessments, or i-Ready. Working the tests into your LCAP metrics is a good way to ensure that you review progress annually and have the data necessary for your renewal.

Keep Calm and Review Your CALPADS Reports!

By Gerald Cockrell, Data and Categorical Program Specialist

November 13, 2017

Given the large amount of data reported for CALPADS Fall I, and the multitude of certification reports available, it can be difficult to determine which data are the most important and where to find them!  To help ease the frustration, we’ve created a list of the most important reports and included detail about the information each report contains and who should be reviewing them.

[table id=1 /]

There are many other reports available as well, but starting with the ones listed in this table will provide a clear view of the data that affect your funding and accountability.  As a reminder, special attention should be paid to FRPM data, EL data, and high school graduate data.

If you have any concerns about the data being reported for Fall I, feel free to reach out to EdTec’s data team to discuss how we can help. You can reach Gerald at gerald.cockrell@edtec.com, or 213.292. 6620 x414.