How can decision-makers gain an accurate picture of pupils' attainment, without subjecting everyone to a barrage of tests? One answer is cohort sampling.
Patrick Kelly finds out what it means.
Cohort sampling - is it another newfangled theory? On the contrary, it has a respectable pedigree. Widespread in England, Wales and Northern Ireland from the late 1970s until the late 1980s, and the introduction of the national curriculum, it is still used in making international comparisons. Indeed much of the comparative data used by governments, including Britain, is based on anonymous cohort samples taken in English schools.
"Cohort sampling answers the question: 'how is the country performing as a whole?'" says Graham Ruddock of the National Foundation for Educational Research (NFER), which has carried out the technique for many years. "And it gives as accurate a picture as can be gained."
Yet government appears resistant to its more widespread use. For the GTC, that's puzzling. "It's somewhat inconsistent to urge that it's of no use, when cohort sampling is the model used in international comparisons of which governments and politicians take so much notice," says GTC Chief Executive, Carol Adams. Speaking at a conference last term, Carol said: "Representative cohort sampling is a highly reliable means of measuring standards at national level, year on year, without testing every child." She cites research which shows that, as they get older, pupils who perform badly in tests become increasingly demotivated because of their past performance, growing more anxious and mistrustful of tests. "Their chances of future success are reduced and the gap between low and high performers is perpetuated," she says.
Further, cohort sampling could provide the answer to some of the major glitches with Sats, says GTC assistant research advisor John Ashby. "The problem with the current system is that it is being asked to do too much," he says. Key Stage tests are not only used to ascertain how pupils are doing; they are also meant to judge how well schools and local education authorities are performing. "Plus being used to determine how we are doing nationally," says John.
This adds to teacher workloads and has a distorting effect on the curriculum and what is taught. "Because the national tests are 'high stakes', in the impact they have on a school's place in league tables, then what you are testing is the ability of a school to meet the test, not what pupils there know and have learned," says John. In contrast, the introduction of cohort sampling would separate what is needed for the national picture and what parents need to know about the school and how their children are doing.
Under this system, tests ask not just one set of questions but a whole range, covering different aspects of the curriculum, although candidates are scored on the same basis. Independent teams – not teachers in schools – carry out the tests. And the whole system is anonymous. No pupils or schools are identified, but the results are compiled to give a representative picture of the performance of whole country.
The international stage
Among the large-scale international projects that use cohort sampling is Trends in Mathematics and Science Study, known by the acronym TIMSS.
This analyses trends in mathematics and science achievement in years 5 and 9 every four years. TIMSS began in 1995, with the most recent study taking place in the northern hemisphere in 2003. TIMSS enables researchers to collect extensive background information about the quantity, quality and content of teaching that can be used to make comparisons between participating countries.
NFER carried out the administration in England on behalf of the DfES. Graham Ruddock, the study's lead researcher for England, says: "Studies like TIMSS increase our understanding of national education systems. Being able to make comparisons between different countries helps to identify our strengths and celebrate them, as well as pinpointing areas of concern and those requiring action.
"But the ultimate reason for doing these studies is that they help to improve the quality of education in our schools and colleges."
Another project is the Programme for International Student Assessment (PISA) organised by the Organisation for Economic Cooperation and Development (OECD), and run by an international consortium.
PISA is an internationally standardised assessment, which is jointly developed by participating countries and administered to 15-year-olds in schools. Assessment first began in 2000 and takes place every three years. Each cycle looks in depth at a major domain, to which two-thirds of testing time is devoted; the other domains provide a summary profile of skills. Major domains were reading literacy in 2000 and mathematical literacy in 2003, with scientific literacy taking centre stage in 2006.
For this cycle, 57 countries are participating in the assessment. In the UK, field trials involved administering tests and questionnaires to about 1,000 students in England, Wales and Northern Ireland in March 2005. The main study, in March 2006, will involve around 150 schools in England, 120 in Northern Ireland, and 120 in Wales, with 35 students per school included in the sample. In Scotland, the survey is being conducted by the Scottish Council for Research in Education (SCRE). Results will be analysed and reported by the international consortium for the UK as a whole, with separate national reports for each country.
*Patrick Kelly is a freelance journalilst.
