Pull-outs Are Making Me Crazy!

I have an interesting problem this year.  I have a class of 25 students, and I have 22 pull-outs.  Many of them are the same kids pulled out for this special service and then that special service.  I have one student who goes to ESL for 45 minutes.  I have two who go to Resource for math for 45 minutes daily, and three who go to Resource for reading, 45 minutes daily.  I have three who see the school counselor once a week for half an hour or so.  I have seven (way too many) who go to Enrichment once a week for an hour.  Then I have six who receive speech therapy for 30-90 minutes weekly.  I have two  half hour blocks every morning, and an hour and a half on Monday afternoon when I have all of my kiddos in class.  There is always someone coming or going.  I have had to post a schedule near my desk of who goes where and when, so that I can pretend to keep track of them.  I am struggling with what I should hold some of these students accountable for.  It seems that, no matter what I teach, someone misses it!  The real frustration is that the teacher next door, (also third grade) has no Resource kids, no speech kids, no ESL kids, and no kids seeing the counselor!  Wow!  Anybody else struggling with kids who are pulled out so much, that it seems like they're never in class?  I would really love to hear your tips and suggestions, if you'd be willing to share!