Achieving consistency in assessment policy and practice: St Christopher’s Multi Academy Trust

Learn more about this key initiative, led by Iain Randall and supported by Scomis, to develop and implement a trust-wide policy for assessment. 

Iain Randall, Virtual Lead, St Christopher’s MAT.  Gary Henderson, School Leadership Partner, Scomis

Session transcript

GARY:

In my capacity as a school leadership partner with Scomis, I work closely with school and MAT leaders such as Iain to understand their challenges and help them address and exploit technology effectively.

The challenge Iain and I are going to discuss today is one that will be familiar to many of you, whether you are from a small primary school or work in a Trust. It concerns consistency and we are going to especially look at consistency in regard to assessment policy and practice. I have worked with closely with Iain on the Trust journey and I have been so impressed by the approach he has taken to deliver the change, but I thought a wider audience would benefit from learning about his experiences. We aim to give you an insight into how, working collaboratively across the eighteen schools in the Trust, Iain has empowered the schools to take ownership of their experience while delivering the outcomes the Trust requires. We will talk about the lessons learnt and how to successfully deliver change to a realistic timeline through good communications and engagement.

So, Iain – in many ways your MAT has an unusual make up, so can you tell us a little bit more about St Christopher’s MAT?

IAIN: We are eighteen schools, and we are stretched across Plymouth, Devon and Torbay. We started with four schools around 2013 or 2014 and at the time I was an assistant headteacher at St Michael’s in Exeter, one of the founder schools. Over time that grew and grew and at the point where it reached the number of around fifteen schools, it was recognised that we needed a bit more support in the middle, so I was seconded to work in a school improvement team at that stage.

My first job was around data and various other bits have been added on since then – I am still looking at assessment and data now. We have a mixture of infant and junior and primary schools. We have mixture of church and non‑church schools and we have a great bunch of heads, and we are very proud of our flourishing community.

GARY: Thinking about our area of discussion today, can you outline to us what the area challenge is and why it was a challenge for the MAT?

IAIN: So, at the time when the schools started coming together, each was pretty much doing its own thing. There wasn’t a MAT way of doing things around assessment of data and one of the things that we realised quite early on is an organisation of our size really does depend on the flow of information between different parties and the schools and the centre and the schools and between ourselves and our directors and trustees needing to know how the schools were performing and so we started off using statutory data that was already available like ResOnline and looking at that and seeing what that had to tell us.

Then we realised we were going to need a termly data collection to make sure we were able to spot where schools might be getting into difficulty or spot where schools were doing well in a particular area. At that time, some schools were using proprietary tracking systems – a couple of schools were using Microsoft Word to track small groups of people. So, in view of that and in view of not wanting to impose something on everyone, we were asking for a manual data collection each term, which I then had to assimilate and try and make sense of and it was a bit of a thankless task at both ends. We needed the flow of information to help us direct school report and schools understood the need to give it, but it was very time consuming. That model led to lots of errors – from typos to misunderstandings – about what was being asked for and we also recognised that within that one of the biggest challenges was: okay, the information is flowing through, but is it accurate? Is it reliable?

So we got to a situation where we had to start tackling the problem at both ends and how were we ensuring that schools were assessing accurately and how were we making sure that the information coming through was being processed accurately and giving a true picture, because one thing I really don’t want to be doing is putting a graph in front of our directors saying the situation is this and that is based on unreliable or inaccurate information. At the beginning of 2019, we needed to standardise our practices and we needed to put something in place that would streamline both the assessment processes and would also then streamline the data collection processes so that it would reduce people’s workload and increase accuracy. Our strategy had two aspects: focusing on our need for information, but also recognising the workload agenda. We were in a situation where every six weeks we were assessing children; every six weeks we were highlighting objectives of the system, and that would add up to a certain value.

Then on we went. It didn’t feel right, and it didn’t feel productive or useful. So the issue became how to balance those two things. The other major challenge was cost. We were very honest with our schools about it because we are looking to save them (and ourselves) money. So that put us in the situation where we really wanted to get out there – to see what was out there and try and find something that was going to meet all of those objectives.

GARY: The idea of “seeing what is out there” is interesting. Why was it that you ultimately came down to focus on SIMS – what were the reasons for that?

IAIN: It was an interesting journey, because I think half of the schools had a package called School Pupil Tracker. Our instinct was that, if half of the schools are using it, we should buy it for the other half and then we have consistency. So that was our hunch – and then one of my colleagues dropped me an email asking if I knew that the School Pupil Tracker would be closing at the end of the year. I don’t remember if my jaw hit the floor or if my forehead hit the table, but I was left thinking “now what?”

So we reached out to various people, and one of the people we spoke to happened to be the head of my children’s school. We found out that they were using SIMS for assessment and tracking, and I think I was a little bit startled by that because my only experience with SIMS had been to take the register. And that was a faff and I could not find my way around and I did not like the look of it. I couldn’t believe that something as clunky as that could meet our Trust assessment needs.

But, having seen from the primary that it was working, I reached out to you and to Craig and we started to discuss it a little bit more. In the meantime, one of the key things for us that I know you want to talk about today is that we were absolutely passionate – we were not going to sit in a Central Office like this and make a decision like this on behalf of everyone. So, we reached out to the schools to ask if heads and assessment leads in schools wanted to be part of the working party to attend demonstrations and it was a bit like something out of Britain’s Got Talent. We had different people coming in and showing their slides and they would ask the questions and we had some online demos which were disastrous and when we talked about it properly as a group, it was decided that SIMS was going to be the way to go.

It met the cost element because we were already paying for it; it met the flow of information element because it was already there in all the schools, so we would be able to harness that information remotely; and it met the workload element because we felt that although we liked some of the packages we thought it would be very easy to spend anything up to £20,000 a year on something that is already made and already done. We felt that was going to shoehorn us into someone else’s way of working and someone else’s intuitive processes and we did not want to be reliant on that. We wanted something with no bells and whistles – we just wanted something simple and what we saw in SIMS was something simple. Simplicity was the USP for us.

GARY: Having reached the point where you have decided you are MIS, the aim must now be consistency. As you say, all the schools were delivering, or assessing in very different ways. So how did you start to bring them together and a common format across the MAT?

IAIN: As I say, we had been tackling the problem at both ends. We were well‑used to the assessment cycle by then, so we were able to publish the dates ahead of time – people knew when they would be submitting data, so we knew when the dates would need to be ready. And we have had lots of moderation exercises across schools, within schools, we have encouraged the use of validation weeks so that often – like the current term for example – Friday the 1st of April is our data collection day. Most schools will have a soft data collection internally the week before and then run their own validation processes, sometimes with support from school improvement, just to check, check, check. Not just accuracy, but also gaps: did we miss anybody? Have there been typos? Did we use the wrong code?

A lot more of our schools now use tests from NFER, for example, to help validate their judgments, which was not standard practice before. So, there has been that growing understanding that we do need that triangulated data and that leaders in school do need to check before they tell me that it is fine to look at. So obviously the bulk of that work goes on in schools: usually in the summer term we do bring in the advisors to help us with our writing moderation, because that is something that some schools find difficult to triage late. That is very much school‑driven and a lot of the heads now will get themselves together in clusters to peer moderate. We have about half a dozen small schools where several teachers will not have a partner teacher to check their books against, so that has been done quite creatively and during the lockdown period we have used programs like Huddle and Teams and other systems which have enabled us to do that remotely. That has been school‑led at our end of things because of the simplicity of the design; we designed a system that would not dictate how the judgments had to be reached. The only thing that the system dictated was when the judgments would go in and what codes we would have that all schools would have used and would have in common, so that we have a common language and expectation. I can remember involving the heads in that, asking how many codes we would need. We settled on working towards expected and greater depth – there were lengthy debates about whether we need a code for a bit more expected, but not quite greater, depth yet. But the heads were adamant they did not want that; they wanted the simplicity. So, it was a very collaborative process.

GARY: You have that collaboration, getting together to reach the end goal of a grade for a child. There has also been some work in the background in making sure that the schools feel supported and about entering that data as well, hasn’t there?

IAIN: 100%. The mark sheets you helped us put together were very easy to use once we have shown people the way around, and we continue to show people the way around. The chief learning points I have taken from this have been primarily that we shouldn’t always go with our first choice, but really explore what is around – as well as involving as many people as possible. Another learning point is that we did not rush; we took a school that didn’t have an are proprietary tracking system and we tested our materials, we tested our mark sheets and our tracking grids, and we tested our training and our codes with one of the largest schools to see how they got on. That gave us the confidence to see that rolling things out was going to be achievable.

It was December 2019 by this point. We then rolled it out to the remaining schools who were about to use their pupil tracker license and left everyone else for a bit – and then all of that was going swimmingly until the pandemic hit and we had to roll it out to the rest of the schools in the throes of lockdown. Because of its online nature, we were able to do that remotely and the team that came together to look at the demos has stayed together. We still meet regularly as an assessment working party. They were involved in agreeing the design of the mark sheets, what should be on there and how they should work and how they should work. They were my critical friends; at times very critical and needing to be so. We now operate on a patch model, so each of the schools has an assigned data lead which may be from their school or any other and they are then the first port of call for training new staff, introducing the mark sheets and supporting people with entering the codes.

From a SIMS point of view, the only data entry issue that people have had is either they sometimes put things in the wrong column – which is relatively easily rectified – or they have struggled to understand and are making a point in time judgment and that is not SIMS’ fault. But it’s been hugely beneficial in terms of sustainability – in terms of upskilling other people and giving opportunity for middle and senior leaders to have a wider trust impact. It has been a really good use of work from that point of view.

GARY: Because you are quite a wide geographically spread map, how have you overcome that with access to different people’s SIMS?

IAIN: All the data leads have remote access to all their schools, so they can just sign in once and then move between schools. And of course, one of the unexpected benefits of the pandemic is that everyone is now used to using Teams; deliver on‑the‑spot bits of training for an administrator in a village in the west of Dartmoor has now become an easy and efficient thing. We might get a message saying “I am struggling with this” – we dial in and that training guidance is provided, and the entire thing might take ten minutes; whereas before, even for their data lead that would be probably a lengthy round trip. So, from this point of view, geography is not a barrier anymore. When I will be doing my term lead SIMS for leaders training, colleagues all over the trust will be trialling into that.

GARY: Are all the schools now on Scomis hosted?

IAIN: Yes, all schools are now hosted.

GARY: So that means you can dial into any of the schools SIMS to look at the data in there. As a data lead, is it easy enough for you to get hold of that data?

IAIN: Yes, it is. The way I tend to work on data collection is that we lock it so that people cannot make any changes. The way we set things up around the reports is at that point in time, leaders in the school can download the reports they want; I will be downloading the same report and using that to give my welfare data about schools that we can use for benchmarking and tracking our overall trust trajectory. This means that it is very efficient and very democratic – I have been very keen from the start of my time with the trust that we democratic with our data and it doesn’t sit with a bearded middle aged data nerd in glasses like me, or just with the Head’s office, but a way that can be shared with governors or the subject leads and the middle leads and the senior leads and the SENCOs. They have access to what they need.

We wanted something that was going to be available to everyone, like my school improvement leaders, so we are all seeing the same information and we are all able to form a perspective and we are all able to move quickly from entry to analysis without having to go through somebody else who would interpret it for you. I am really keen on the visuals and how things look and that means we are able to put the information in people’s hands in a way they can quickly understand and assimilate.

GARY: For me, the phrase that comes to mind is: “if you look after the pennies, the pounds will take care of themselves.” In this scenario, the “pennies” are how the children are achieving. So how have you used your MIS to feed data back to the classroom teacher?

IAIN: What is helpful is that, even as teachers are putting data on, because it has got all the children’s learning journey in front of them, they can see for themselves how the child’s progress is tracking – that is giving them instant visual feedback. We also use the tracking grids which is a one‑pager – one-pagers are considered trendy currently. That includes the children’s starting point, with their current data on a matrix basis and really draws attention to who is at risk of falling behind; whether they are slipping back from expected. Now that they have that data, a lot of teachers can pull that tracking grid and start preparing for their pupil progress meetings. It is standard practice in our schools to talk about the data and it means that that preparation is done, because we don’t have to. Before they were having to sit with a piece of paper and make a Venn diagram – who are we worried about and who are we not worried about – there it was, make a few notes, off to PPMs and able to get into conversations about the children at risk of slipping behind. What needs to be in place for them? The dialogue is not about the “who”; it’s about the “what next”. It is a time saver and so teachers mainly use the marking sheets and the tracking grids, but we do have class level reports that they can use. The trouble with that is that the group sizes get smaller, meaning that the percentages are perhaps not so beneficial.

GARY: The small cohorts with the percentages become meaningless, don’t they?

IAIN: What is really powerful with that is, because we have 3500 pupils, when we pull the data together we are able to give that whole Trust picture and then our scores can benchmark against each other and then we have an overall sense of how we are progressing.

GARY: What other tools have you used to bring that data together?

IAIN: As you know, I am a bit of an Excel geek, so I usually pull the reports down because they are exportable as Excels and then I have a master sheet that once that data is slotted in, it starts updating the percentages for each year group across the Trust. I can quite quickly see how that is coming together and then I use that to generate a range of graphs and charts that I put into my reports.

There has been a little graph that you have created called the Manage Reporting Service which is still quite new. We are still learning what it is for and where it fits in. It is based in Power BI and that looks at SIMS for all our schools every night, pulls a range of data through and then gives us the ability to interact with that data in a more refined way: we can filter by school, by pupil premium, and by EHC plan. It really does give a quick visual picture. If a trust lead like me – eighteen schools, 3500 pupils – needs to answer a question of how many EHC students we have in Plymouth, I can do that without having to go into each individual SIMS. It’s not just the assessment data it pulls through: it pulls through the exclusions data which has been useful for me because I have to report on exclusions. It also pulls through staff data. I know the HR team at our Trust make huge use of tracking staff absence and other staff indicators to help them find the right support where it is needed.

So that is the only bit we are paying extra for, but relative to our size it is a very small amount for our pupils. I know the plan is that heads in our schools will be able to see our profiles. That is going to start changing everything. I think there is a risk in an organisation like this that people feel the job is done when they put the data on, and they don’t see where it leads to and sometimes that can lead to gaps.

One example is the Manage Reporting Services pulling through the EAL data and that showed up that about 40 or 50 children do not have a language at all and we have those gaps we are keen to see we close. Some of the data, like assessment, is only ever going to be a termly thing, but a lot of the data, like exclusions and attendance and staff absence and wellbeing, is daily data. So, I am excited that next year we will be able to put those in the heads hand and make use of that intelligence themselves.

GARY: Going forward – you have done this work, you have it embedded, how do you ensure it stays embedded? I think in some ways you have answered that. Then, how do you see the next steps for the MAT? Where do you want to start taking things in another direction, which you sort of touched on with the MRS?

IAIN: I think that live interaction with live data is a logical next step for us around the things that are relevant. At a personal level I am looking to hand the project over. I think we may well have internal capacity now within our assessment leads to take more responsibility for running the project.

In terms of how the data goes on and how it looks in SIMS, my big vision is to make no change at all. One way we keep things embedded is to stop fiddling and adjusting. I think, really, it is about squeezing as much knowledge and expertise out of yourself over the next 12‑15 months into those data leads so we can start creating more bespoke reports. I think we have our fixed ones, but it would be great to have internal capacity to say, can we produce a table that compares those different sources of information in a report. It would be great to be able to do that without having to go back to you. Building the internal capacity, we have stuck with this system now for two‑and‑a‑half years and we have got a lot of people trained in it and we have the core group of data leads who know that little bit more and are able to upskill people when they join. Fundamentally the product itself, once you know your way around and the route for things to click, click, click, is ultimately quite simple to use, so it is much easier to embed something that is simple.

GARY: Yes.

IAIN: That is my big vision. A bit boring really!

GARY: What I like about St Christopher’s you have not gone for the big bang thing. The trouble with explosion is that it can knock people over – what you have done, caringly, is carry people with you. And I think you have done that so well. It is brilliant.

So, we have had questions coming in, if we go in the questions that we have had so far. If there any questions which we don’t answer before the end of the session, we will catch up later and send their answers to them.

In your view, as a MAT building that local expertise, what are the benefits of a MAT? You put a lot of time and effort in building the local expertise with the assessment needs, how has it benefited the MAT?

IAIN: One fear that schools often have coming into MAT is that they will lose all autonomy and be told what to do, so the fact that it’s been an organic and a collaborative process has helped to offset that. Having the data leads, we are open to feedback from schools and able to shape and to reshape things, based on feedback. We have been able to respond very quickly if there have been gremlins – because the fact that we have been engineering it ourselves means that it has been hard work and there have been gremlins. We have not got everything right, but it meant that we have been able to pick those things up quite quickly. As I said, it means it is sustainable going forward. But a real positive is, it means there is a group of staff there whose involvement and awareness and understanding of the MAT is not a big faceless organisation that makes us to do things, it is something that they are a part of and something they can contribute to. As we come together as a group of eighteen schools, we are starting to see that happening more often in our areas. Some of my team are teachers and some of my team are senior leaders, but they all know that they have a wider contribution to the group. And I just think it is a huge benefit to everybody.

GARY: That leads on to the next question, which is: how applicable is this approach to other developments that MAT’s intending to implement?

IAIN: Well, it is very applicable. It is always on our agenda to seek unlocking capacity from the schools. We do have a school improvement team – all very skilled and knowledgeable individuals – but we are a finite resource and there are eighteen schools. It is not a secret that we want to improve our schools and in several of our schools our grading requires improvement. Currently we are benefiting from intensive support and waiting for that phone call, so we cannot do everything from the middle.

We need to build networks of people, used to working with each other, who recognise where the strengths and the talents are. This year we have several staff working as subject champions for their areas, working as lead practitioners to get out, to give support to each other at the peer level. We have a long-standing peer review model, where heads work with the team that I work for and support schools we valuation and improvement, so, it is very in keeping with who we are and who we want to be and how we want to work. And not just within the school improvement side, there is more collaborative working going on between the office staff. And in other functions of the business too. It is very applicable. I would say if other Trusts are thinking it of going down the route you have thirty-five schools or five, it will be good to be thinking about who is leading on it for that school and who is the party to drive this? As anything that depends on one person is vulnerable.

GARY: Thank you for your time and sharing your insights.