The Folly and Frustration of Evaluating Schools and Staff Based on the Progress of Students with Significant Social, Emotional, and Behavioral Challenges

Understanding the Student, Home, and Community Factors that Impact Challenging Students

Dear Colleagues,

“The ultimate measure of a man is not where he stands in moments of comfort and convenience, but where he stands at times of challenge and controversy.” ---- Martin Luther King, Jr.               
Introduction

Hang on, folks.  Today’s message is probably going to wander a bit, but I promise that it will (hopefully) make sense by the end. . .

When I was, for thirteen years, the Director of the U.S. Department of Education-funded State Personnel Development Grant (SPDG) for the Arkansas Department of Education, I would often tell my staff (courtesy of John Wooden):

“Never mistake activity with achievement. Activity doesn’t count unless outcomes result.”

What I was trying to communicate was that I was less concerned about the number of phone calls, meetings, on-site school consultations, and student recommendations that they made each week, and more focused on whether those activities actually impacted students’ academic and social, emotional, behavioral (short-term) skills, mastery, and application outcomes, and (long-term) proficiency and independence outcomes.

And while student outcomes clearly were “Job 1,” I also knew that we needed to similarly look at and facilitate how staff and schools were learning, mastering, and becoming more independently proficient in:

  • Organizational development—including resource mapping and development, capacity-building and sustainability, and systems-level support to schools, staff, and students
  • Effective school and schooling—including the use of scientifically- or research-based practices at the administrative, curriculum and instruction, progress monitoring and evaluation, and multi-tiered (i.e., prevention, strategic intervention, and intensive need) service and support levels
  • Professional development and staff evaluation—including supervision and mentoring, and teacher/educator effectiveness, accountability, and evaluation
  • Multi-tiered academic instruction, assessment, and intervention— including positive academic supports and services
  • Multi-tiered positive behavioral support systems—including attention to school safety, school and classroom climate, effective classroom management, and student health, mental health, and wellness
  • Multi-tiered systems of support—including problem-solving teams, consultation processes, and data-based functional and diagnostic assessments leading to effective instructional modifications and/or academic/behavioral interventions
  • Parent and community outreach and involvement—including needs assessments, training, support, capacity-building, advocacy, and the braiding of school and community services and supports
  • Data management, evaluation, and accountability—including the formative and summative tracking of system, school, staff, and student outcomes

[Click to see PART I and PART II of my recent series on Why Strategic Planning Must be Done by Every School, Every Year]


The point here is that:  Students’ school-specific outcomes largely occur because of effective, multi-tiered school and schooling practices.

At the same time, the Elementary and Secondary Education/Every Student Succeeds Act (ESEA/ESSA) accountability goals of the U.S. Department of Education and our state departments of education do not factor in the individual and collective impact of:

  • Student Factors (e.g., their history, ecological status, potential, actual ability, and motivation to learn)
  • Home Factors (e.g., the impact—on students’ school readiness and performance—of parental education, income, presence, parenting, stability, and support)
  • Community Factors (e.g., the availability of well-paying jobs and appropriately-funded schools, health and mental health personnel and resources, social and related service agencies, after-school and youth-support programs)


And there are some students who are not going to succeed in school—the way we all want—because of these factors.

Sadly, framed by ESEA/ESSA, the U.S. Department of Education and our state departments of education do not want to formally acknowledge this in their assessments of districts and schools.

That is, regardless of individual student conditions, these organizations still focus predominantly on “macro-analyses” of school, grade-level, and/or group or “sub-population” data.  And, these analyses are not sensitive to, and the education departments do not factor in (e.g., through covariant analyses) the challenging student, home, and community factors that sometimes negatively impact these data.

I have personally seen this short-sightedness.

Because my Department of Education work occurred largely on-site and in long-term professional development and consultation relationships with individual schools and districts across Arkansas (and our country), I was intimately aware of how Student, Home, and Community factors undermined student performance.  Moreover, most of these factors could not be reasonably influenced within the school.

In fact, I can remember the “horror” of the State’s Director of School Improvement when I returned from a multi-day consultation with one of our “lowest performing” ESEA schools, and I said:

“Leave them alone.  They are doing the right things given the conditions they are facing.  If they did anything more or different, it would completely screw up their progress.”

My point was that the additional progress that we wanted for the students in this school was NOT going to occur through additional educational intervention.  Instead, it was going to occur through collaborative community, home, and student partnerships.


Trauma, Transitions, and Teaching

“Problems are not stop signs, they are guidelines.” ---- Robert H. Schuller

One of the great things about consulting around the country (and world) is that, in different districts or schools (or countries), I get to do a wide range of different things.  This is what makes my “job” fun, and I get to learn new things all the time.

In some districts and schools, I am helping at the organizational (including community) and systems level.  In other districts and schools, I am assisting to address the intensive needs of individual students (and parents) . . .  And everything in between.

Indeed, here is a continuum of what I do from system to school to staff to student to home and to community:

  • ESEA/ESSA Strategic Planning and Preparation
  • School Improvement or School Turn-Around Planning and Execution
  • Leadership, Teaming, and PLCs
  • Staff Evaluation, Supervision, and Coaching
  • Differentiated Academic Instruction and Academic Interventions for Struggling Students
  • Multi-tiered (RtI/MTSS) Services, Supports, and Program
  • School Discipline, Classroom Management, and Student Self-Management (SEL/PBSS)
  • Disproportionality Relative to Office Discipline Referrals or Suspensions/Expulsions of Students from Minority Backgrounds or Students with Disabilities
  • Social, Emotional, and Behavioral Interventions for Challenging Students
  • Formative and Summative Evaluations, Data Management Systems, and ESEA/IDEA Reporting


This past week, I was consulting at an elementary school where I have worked for the past four years—helping them develop interventions for a number of very challenging students.  This very rural school has fewer than 500 students and is in a small district that has access to some community-based and regional resource center services.  Critically, these services and supports are not enough given the intensity of student need.

As with any school, many of the students are doing exceptionally well—given the conditions.  But from an academic and social, emotional, and behavioral perspective, the most needy students are exceptionally needy.

Here are but three of the students that I worked with this past week:

Jaynie.  

Jaynie came from another district to this school after a year in kindergarten—where she was identified as a student with a learning disability.  While she completed her kindergarten year and entered “our” school as a First grader, her academic skills were so low and she was demonstrating such high levels of work-related emotionality that her mother agreed with the school’s recommendation to return Jaynie to kindergarten in October.

Just prior to that meeting, a Cumulative Record Review revealed that, during kindergarten in her previous school, Jaynie had had three different teachers (one of them a long-term substitute).  And while there were no attendance issues and her report card grades were low in most academic areas, it did not appear that she was considered for retention—even though she was young for grade.

At home, Jaynie is being raised by her now-single mother who has some documented mental health issues herself.  Community mental health supports are limited at best.

Just this week, Jaynie was screaming in the classroom, refusing to do her work, and threatening peers—all requiring one-on-one support and supervision.

Aaron.

We have been working with Aaron, now 9-years-old and in Third Grade, since First Grade.  Across this timespan, Aaron has demonstrated a significant range of social, emotional, and behavioral problems that have required office discipline referrals, school suspensions, the need for him to spend time in our “Opportunity Room” with one-on-one supervision (including parental supervision), and one-on-one instruction with a special education teacher.

In the classroom, Aaron will (a) make noises (routinely getting louder when asked to stop), (b) demonstrate inappropriate behavior immediately after teachers have asked other students to stop the same behavior, (c) refuse to follow directions, (d) make numerous negative self-statements, (e) be aggressive with both peers and staff, and (f) engage in manipulation, testing the behavioral limits, inappropriate “negotiations” to get out of expected work or behavior, and other antisocial interactions.

During Second Grade, Aaron attended another school district, but his behavior was so bad that his parents home-schooled him (instead of a suspension) during the second half of the year.

In Third Grade, his inappropriate behavior (including a number of staff assaults) escalated such that the principal referred him into the Court system.  Part of this arrangement included transporting Aaron to the Court’s “Quiet Room” instead of suspending him.  Numerous supervised stays there have not change his behavior.  In fact, Aaron recently assaulted another school staff member, and has now been suspended from school by the Court.

A Court-ordered psychological assessment, confirming previous school assessments, diagnosed Aaron as Oppositional Defiant with Attention Deficit Disorder, and Anxiety Disorder (Not Otherwise Specified). Intellectual and achievement testing could not be completed because Aaron refused.  And new information was uncovered that Aaron was hospitalized after his brother hit him in the head with a baseball bat at age 2 and a half.

Aaron’s parents deny that he has any behavioral problems at home, and yet, the Court-ordered psychological assessments they completed contradict this.  The regional resource center does not have a program for Aaron (and believes he is “socially maladjusted).”  The Court cannot place Aaron into a juvenile residential program until he is at least 10-years-old.

All of the educational partners acknowledge that the Court suspension and off-site special education services to-be-delivered will not provide Aaron with the intensive cognitive-behavioral therapy and classroom interventions that he needs (these school-based therapeutic services are unavailable).  Moreover, the partners are willing to place Aaron into a therapeutic residential program (my strong recommendation), but are deferring to the Court’s “placement.”  Finally, my ongoing calls for a comprehensive neurological assessment have not been addressed.

Betsy.  

Betsy is a Fifth Grader who we have been working with for over two years.  Betsy’s most problematic behaviors include: (a) “shutting down” during class, and putting her head on her desk; (b) periodically lashing out at peers when she is frustrated; (c) getting emotional—particularly when she doesn’t get her own way; and (d) not getting her work assignments completed. 

Betsy was socially isolated relative to her peers, she would often read books of her own choice—rather than completing assigned work, and she made frequent trips to the Nurse.

Betsy entered the school in the middle of Third Grade as part of a court order where she was removed from her mother and step-father’s home and relocated to her biological father’s home.  The removal occurred because of the presence of drugs in her mother’s home, Betsy’s observation of domestic violence and sex acts that including whipping (Betsy called 911), and the resulting direct physical abuse that Betsy experienced.

Betsy’s biological mother had a history of bipolar mental health issues, and Child Protective Services and Family Court was involved in her case.

Given the history of trauma evident in Betsy’s past and the social, emotional, and behavioral concerns, individual therapy with a psychologist skilled in trauma and cognitive-behavioral therapy was recommended. While she was receiving counseling from the school counselor and a regional resource center social worker, neither of these professionals had the skill or time to meet Betsy’s intensive needs.

Student Analysis and Implications

Jaynie, Aaron, and Betsy—each for somewhat different reasons—are not coming to school each day “ready for Primetime.”  Their classroom engagement (if they actually get to the classroom) varies from day-to-day.  Their teacher and peer interactions range from neutral to nonexistent to negative.  Their “response-to-educational-intervention” is slow, limited, inconsistent, or rejected.

They each have significant academic deficits, and they are not learning.

They all will take the state proficiency tests at the end of the school year, and each of them will fail.

And if there are more Jaynies, Aarons, and Betsy’s in the school (and there are many), many grade-levels will “fail” on the state proficiency test, and the school will find itself in ESEA improvement status.

This is not to excuse the non-proficient status of the school, but to understand it.

But it also is to understand (as above) that education-only interventions will not solely increase the academic proficiency of this school, its grade-levels, or its individual students.

And it is also to recognize that many schools, districts, and communities have resource and expertise gaps and limitations relative to addressing the immediate and comprehensive needs of these students.


Intervention that Impacts Trauma

Education has just recently gotten into the “trauma business.”  And while districts and schools are recognizing the impact and educational implications of trauma, as a neophyte in the process, they do not fully understand it from a neuropsychological and—especially—multi-tiered psychoeducational intervention perspective.

Moreover, in their desire to help these students, to increase their success in the classroom, and to enhance their staff’s confidence and ability to teach, many districts and schools have accepted, purchased, or implemented trauma programs “off-the-rack.”

And yet, many of these programs:

  • Have not been extensively field-tested and empirically-validated in a range of districts with a range of students with different traumas
  • Are not based on relevant and meaningful neuropsychological and psychoeducational science-to-practice principles and practices
  • Are or are seen as “yet another program” that staff need to do in addition to this program, and that program, and this additional program, etc.


You see, trauma has become a big business in education, and the entrepreneurial players see districts and schools as “a market” for their (not-yet-validated) perspectives and programs, while the well-meaning “do-gooders” naively believe that they hold the key to solving the problem.

The Bottom Line is:  Student trauma is unique to each student.  Thus, the students and situations require diagnostic assessments that are then linked to strategic and/or intensive clinical, school, and home interventions. 

These analyses and interventions then can be organized into those delivered to (a) individual students, (b) groups of students (based on common needs), (c) grade-levels of students (given the developmental differences across some grade-levels), and (d) all of the students in a school.

While described from the “inside-out,” the continuum above reflects the multi-tiered nature of most intervention systems.

But it is critical to note (as above) that the neuropsychological science underlying multi-tiered interventions for trauma involves the same science as multi-tiered approaches to school discipline, classroom management, student self-management, emotional self-regulation, teasing, bullying, and physical aggression.

[Recent Blog: “Effective School-wide Discipline Approaches: Avoiding Educational Bandwagons that Promise the Moon, Frustrate Staff, and Potentially Harm Students. . . Implementation Science and Systematic Practice versus Pseudoscience, Menu-Driven Frameworks, and “Convenience Store” Implementation”]

To summarize again:  We must understand that some student’s “academic difficulties” intersect with their social, emotional, and behavioral difficulties.  And these difficulties must be understood and addressed by looking at the student, home, and community factors that are underlying the difficulties.

Moreover, it is inappropriate to evaluate, down-grade, and—in essence— blame our schools for social, emotional, and behavioral issues that directly impact students’ academic outcomes.

A Personal (and Professional) Note 

“In order to succeed, we must first believe that we can.” ---- Nikos Kazantzakis

While I began the section above by talking about how “fun” my consulting job is, my involvement in cases like those of Jaynie, Aaron, and Betsy is frustrating, heartbreaking, depressing, and paralyzing at times.

It’s frustrating because of the limited resources in many geographic areas across the country, but also because:

  • These areas are not funding and/or allowing their professionals to get the strategic and intensive clinical training that they need;
  • The schools, districts, regional resource centers, and community professionals are not collaborating on an ongoing, consistent, and effective basis; and
  • There are political, policy, procedure, and practice differences that establish engagement and enactment barriers that interfere with the school, regional resource center, and community-based collaboration.


It’s heartbreaking because of what these children have experienced, are experiencing, and will likely experience in the future.

It’s depressing because the prognosis for these children is so bleak and their needs are so great.

And, it’s paralyzing because you sometimes don’t know where to start, what to do, and whether you can overcome the system gaps, the staff limitations, and the students’ (sometimes) day-to-day crises.


Given all of this, it would be easy to disengage, move into a protective mode, and/or lower your expectations.  And there are times that I, too, throw up my hands in despair and disbelief.  But at some point, my sense of right, advocacy, persistence, and belief that change will occur returns. . . and we start “working the problem” again.

Most educators are “wired” this same way.  That is, when confronted with seemingly insurmountable situations, they kick it into gear and take on the challenge.  They believe.  They work the problem.  They facilitate change.

And while this is all honorable and remarkable, we need to take care of these professionals.  More specifically, we need to make sure that our educational “first responders” get the support and care that they need. . . so that they can continue to give our most challenging students the services, supports, interventions, and the hope that they need. . . at full strength.

Yes. . .  one of the social-emotional supports that Jaynie, Aaron, and Betsy need is hope.  Even when they have given up, we need to raise them up by telling them, showing them, and proving to them that they can succeed. Sometimes, we need to be their hope . . . until they can bring their own hope to the table.


Summary

Thomas Edison said, “Many of life’s failures are people who did not realize how close they were to success when they gave up.”


I have yet to work with a school, across this vast country, that does not have a group of students who walk into school every day with significant and disruptive challenges that are not of the school’s making.

We need to understand the student, home, and community factors that underlie their challenges, and recognize that—ESEA or not—we must help these students to move past, resolve, cope with, or compensate for these factors.

This takes the right resources, applied in the right ways, implemented in the right places, with the right levels of intensity.

But it also takes hope, belief, and a realistic level of optimism.

It is not easy, but it is necessary.  We must “work the problem” rather than “blame the victim.”

“It always seems impossible until it’s done.” ---- Nelson Mandela

I hope that this message has touched you in some way.  The work that you do is important, and you are being successful—even if it does not always seem that way.

How many of us have had students return to visit us. . . years after they left or finished school?  These are the students that we never felt we “reached”. . . that we never felt successful with.  And yet, they are at our door to thank us for what we did for them, and how we significantly impacted their lives.

Multiply the one or two students who return by one or two hundred (or more).  Because those one or two students represent the hundreds of frustrating and challenging students that most of you have positively impacted. . . you just may not know it.


If I can help you to impact more of your students, I am always available by e-mail or phone.  Let me hear from you.  Let me know how I can help you, your colleagues, your school, or your agency/organization go to the next level of excellence and impact.

Best,

Howie