Sociologies of Rankings

Breve artículo de José Ossandón para journal online griego: Re-Public cuyo tema puede ser de eventual interés acá….

There is no doubt: rankings are everywhere. We are surrounded by thousands of lists measuring a vast array of variables such as: competitiveness, productivity, quality of life, risk, and even happiness. Some globally famous examples are: Fortune 500, the FICO Score, The Times Higher Education World University Ranking, The Programme for International Student Assessment (PISA), Global Competitiveness, and the UNDP Human Development Index. Together with the growing relevance of rankings, in the last years there has been a multiplication in the modes of approaching these objects within social sciences and particularly sociology. In the following paragraphs I distinguish six different sociologies of rankings, and to finish, in the last part, I give a brief comment about some of the political consequences connected with each of these levels.

1. Quantitative scales have been part of social sciences since their beginning (Hacking 1990). Sociologists have since then participated in the production of new modes of listing different social units (such as: countries, firms, universities, cities and people) numerically. Their work in this context has had to do mainly with the delimitation of the object of analysis (for instance, the “level of productivity of different researchers” or “poverty in different countries”), and how to translate them into dimensions, indicators and variables that can be measured. Actually, a very central proportion of the education in methods for undergraduate students even today has mainly to do with the development of these types of tools. Despite its eminently expert character, sometimes discussions regarding how to measure a social index can be the object of heated public debates (for instance when there is no agreement about how to measure inflation, Neiburg 2006, or the risk associated with the bonds of a specific country).

2. The tools of sociology are also central in the analysis of social rankings. This analysis can have a “descriptive” or a “structural” character. In the first case, the main variables are described (for instance by building averages or distribution charts), while in the second case, their distributions are correlated to other variables such as age, sex or socio-economic origin. Structural analysis, even though it does not have the hegemonic position it used to have (Savage 2009), is part of a long and respected tradition that goes from Durkheim’s “Suicide” to Bourdieu’s “Distinction” with Coleman’s famous “Education Report” in between. These types of methods, as everyone who has attended statistics courses knows, are a world in its own, going from the simple cross tabulation tables to the more sophisticated regression analyses. The structural analysis, given its scientific character, tends to inform important decisions concerning social policies. Some topics where this is particularly relevant are “development”, “inequality” and the association between household conditions and educational outcomes. It is important to keep in mind, however, that these methods are used by researchers inspired by multiple kinds of theoretical or political principles (such as “public choice”, “social cohesion” or even the rather leftist critical structural sociology).

3. Rankings are also the object of criticism as they reduce complex social practices to abstract indicators that are hierarchically comparable. This type of criticism is generally produced by defenders of qualitative and ethnographic methods to approach the social world. In this context, but at a more general level, rankings are criticized because they do not only poorly represent what they observe, but because they also violate the observed social practices, insofar as the results of measurements end up imposing themselves on them (Miller 1998). For example, it is possible to argue that the complexity of a classroom is not just poorly represented by international tests such asPISA, but also social policies that react to such simplifications act upon the everyday practice of teaching and learning. This kind of discussion is related with controversies about how determined social spaces should be protected from quantification (as it is done, for instance, in those countries where it is not permitted to numerically evaluate students up to a certain age).

4. Rankings can be analyzed as a type of “generalized media of communication” (Chernilo 2006). Rankings, like Money, are abstractions that enable comparison and transmission of specialized and complex information in a simple, or at least comparable, way. Rankings in this sense act like a music ripper that transform analog sound into digital music, simplifying and at the same time increasing the possibilities of transferring and re-combining the new information (Ganβmann 1988, Luhmann 2007: Ch. 2). Rankings make very different situations comparable (such as the academic results between a department of art and a department of engineering, or the risk to invest in China or on Easter Island), and, in this way, they increase the options of exchange among multiple actors and organizations (for example: among investors and producers without a common language). However, while simplifying the world, rankings enable the emergence of new levels of complexity. For instance, more auditing doesn’t necessarily make administration simpler (Power 1997), or the proliferation and extension of forms of risk assessment in finance, that have been essential in the development of asset-backed securities, have in turn increased the risk of the global financial system (Poon 2009).

5. The process of producing rankings can also be studied as a social process in itself. From this perspective what has been called “commensuration” (Espeland &Stevens 1998) or also “investment in forms” (Thevenot 1984) becomes very relevant. These concepts refer to the process needed to produce and apply a ranking, which includes the “technical” task of producing the instruments of measurement, as well as the slow “harmonization” needed to make different units comparable (Desrosieres 2000). Today these processes largely transcend national discussions and are an important part of producing new global institutions (Barry 2006). Within this context, sociologists are spending more and more energy following the production and consolidation of rankings (Sabaté & Tironi 2008).

6. Finally, once stabilized, rankings are transformed into a new social thing that, despite the contingency of its production, appears like a “black box” for the actors that confront them (Latour 2005). In this sense, it is essential to study how rankings are “enacted” (Mol 2002; Law & Urry 2004; Ossandon 2009), or to follow them as “social objects” that transform the world they inhabit. One of the most discussed issues in this context concerns those cases where the introduction of quantitative devices makes what they measure more “calculable” or “competitive”, questioning the limit between what is observed and the instrument of observation (Barry & Slater 2005; Espeland & Sauder 2007). At the same time it is more and more recognized that rankings are not only disciplinary devices, but they also enable the emergence of new collective actors – such as the “European citizens” (Law 2009), occupational categories (Desrosieres et al 1983) -; new markets (Poon 2007), and even new “concerned groups” emerging through the side effect of a specific type of statistical device (Callon 2007) – such as those that are excluded from a specific form of ordering customers in the contemporary banking system (Leyshon & Thrift 1999).

To sum up, rankings are multifaceted objects and it is possible to approach them from very different perspectives. It is important to consider that from each of these points of view different types of political questions are raised. As already mentioned, from the first type of analysis there can be public controversies about the quality of instruments used to measure social variables. Second, collective binding decisions are closely entangled with structural quantitative analyses (suffice it to think of the Stern report and its impact on the discussion of global warming). Third, also of a political nature, those elements of social life that should be “defended” from quantitative comparisons must be defined, as well as the re-introduction of the reactive or collateral effects of ranking in the definition of new policies. Finally, rankings are central agents in contemporary “hybrid forums” (Callon et al 2001), where controversies regarding them are not just about different methods but about the responsibility in the production of new natural and social “entities”, or what has been called a “political ontology” (Mol 1999).

Barry, A. (2006) ‘Technological Zones’, European Journal of Social Theory, 12 (1), 155-174.

Barry, A. and Slater, D. (2005) ‘Technology, politics and the market: an interview with Michel Callon’, in A. Barry and D. Slater (eds) The Technological Economy,London: Routledge.

Callon, M. (2007) ‘An Essay on the Growing Contribution of Economic Markets to the Proliferation of the Social’, Theory, Culture & Society 24(7-8): 139-163.

Callon, M.; Lascoumes, P.; Barthe, Y. (2009) Acting in an Uncertain World: An Essay on Technical Democracy, Boston, MIT Press.

Chernilo, D. (2006) ‘La teorización de la coordinación social en sociedades diferenciadas: La teoría de los medios simbólicamente generalizados en Parsons, Habermas y Luhmann’, in Farías, I. & Ossandón, J. (eds.), Observando Sistemas. Nuevas aportaciones y usos de la teoría de Niklas Luhmann, Ril Editores – Fundación SOLES, Santiago.

Desrosieres, A. (2000) ‘Measurement and its Uses: Harmonization and Quality in Social Statistics’, International Statistical Review, 24 (2), 312-343.

Desrosieres, A.; Goy, A.; Thevenot, L. (1983) ‘L’identité sociale dans le travail statistique : la nouvelle nomenclature des professions et catégories socioprofessionnelles’. Economie et statistique, 152, 55-81.

Espeland, W. and Sauder, M. (2007) ‘Rankings and Reactivity: How Public Measures Recreate Social Worlds’, American Journal of Sociology, 113(1): 1-40.

Espeland, W. N. and Stevens, M. (1998) ‘Commensuration as a Social Process’, Annual Review of Sociology 24, 312-43.

Ganβmann, H. (1988). ‘Money – a symbolically generalized medium of communication? On the concept of money in recent sociology’, Economy and Society, 17(3), 285–316.

Hacking, I. 1990 The Taming of Chance,CambridgeUniversity Press,Cambridge.

Latour, B. (2005) Reassembling the Social. An introduction to Actor-Network Theory,Oxford:Oxford.

Mol, A. (2002) The Body Multiple. Ontology in medical practice,Durham: Duke University Press.

Mol, A. (1998), ‘Ontological Politics. A Word and Some Questions.’ The Sociological Review, 46: 74–89.Law, J. (2009) ‘Seeing like a survey’, Cultural Sociology, 3(2), 239-256.

Law, J. and Urry, J. (2004) ‘Enacting the Social’, Economy and Society 33(3): 390- 410.

Leyshon, A. and Thrift, N. (1999) ‘Lists come alive: electronic systems of knowledge and the rise of credit-scoring in retail banking’, Economy and Society 28(3), 434-466.

Luhmann, N. (2007) La Sociedad de la Sociedad, Herder-Universidad Iberoamericana, Mexico DF.

Miller, D. (1998). ‘A Theory of Virtualism’. En Carrier, J. (ed.) Virtualism: A New Political Economy, Berg,Oxford.

Neiburg, F. (2006) ‘Inflation: Economists and Economic Cultures in Braziland Argentina’, Comparative Studies in Society and History 48(3): 604–33.

Ossandon, J. (2009). The Enactment of private health insurance in Chile. Unpublished PhD Thesis, Goldsmiths, University of London.

Poon, M. (2009) ‘From new deal institutions to capital markets: Commercial consumer risk scores and the making of subprime mortgage finance’, Accounting, Organizations and Society 34(5), 654-674.

Poon, M. (2007) ‘Scorecards as devices for consumer credit: the case of Fair, Isaac & Company Incorporated’, Sociological Review, 55(2), 284-306.

Power, M. (1997). The Audit Society: Rituals of Verification,OxfordUniversity Press,Oxford.

Savage, M. (2009). ‘Contemporary Sociology and the Challenge of Descriptive Assemblage’ European Journal of Social Theory 12(1): 155-174.

Sabaté, J. & Tironi, M. (2008). ‘Rankings, creatividad y urbanismo’, Revista Eure, Vol. XXXIV (102), 5-23.

Thevenot, L. (1984) ‘Rules and implements: investment in forms’, Social Science Information, 23(1), 1-45.

Post a comment or leave a trackback: Trackback URL.


  • Javier Hernandez  On October 18, 2011 at 5:38 pm

    Me parece extraordinario tematizar esto sociologicamente, y creo que aqui se hizo muy bien.

    En particular me gustaria comentar sobre el rol de los rankings en educacion, sobre todo los rankings universitarios. Un par de rankings lograron de alguna manera resolver el complejo desafio de cuantificar y hacer comparable algo tan abstracto como la calidad de la educacion provista por toda una universidad y tras diversas publicaciones, lograron consolidar sus rankings en el contexto de sociedades que cada vez valoran los indicadores y los esfuerzos de ese tipo. Ello, evidentemente requirio que en el proceso de operacionalizacion se echara mano de todos aquellos elementos que son propiamente cuantificables (numero de publicaciones, metros cuadrados, proporcion academicos/alumnos, etc), reducienciendo gran parte del concepto de calidad de educacion a dichos elementos.

    Hasta ahi no veo mayores problemas, sin embargo, con el correr de pocos anhos, estos rankings han pasado a convertirse en un standard de calidad de educacion, vale decir, para ser una universidad de calidad se debe cumplir con las mediciones de estas instituciones. Se han convertido exactamente en dispositivos que en forma de black box dictan que deben hacer las universidad para ser consideradas de calidad, olvidando que son en esencia una reduccion de la realidad.

    El uso acritico de estos rankings, desconociendo ademas que en ambos, en mayor o menor medida, subyace el modelo norteamericano, o si se quiere, anglosajon, de universidad, en desmedro de otros modelos igualmente validos y/o exitosos como el frances o el aleman. Finalmente, las universidades, ya insertas en un sistema universitario global, buscan mejorar sus posiciones en estos rankings, y algunas comienzan a parecer empresas de produccion de papers en lugar de casas de estudio enfocadas al intercambio de ideasy la pedagogia.

  • joseossandon  On October 19, 2011 at 9:38 am

    uff. Totalmente de acuerdo Javier. Es un gran tema y que probablemente nos toca muy de cerca a todos los que trabajamos en universidades … Justo esta semana leí el artículo de L. Karpik en este libro:
    Ahí el estudia el particular caso de las ciencias sociales en Francia, luego del impacto de nuevos sistemas de financiamiento de la investigación. Según sugiere se estaría pasando de un sistema basado en la evaluación de la calidad de una institución vía peer review a otro donde las agencias considerarían los rankings de las publicaciones de cada departamento (esto último de todas formas pero más indirectamente basado en la evaluación peer review de cada journal). Según entiendo algo muy parecido está pasando en UK con los cambios al RAE (Research Assessment Exercise). Aunque de esto sabe mucho más F. Camerati. El punto de Karpik es que ambos serían modos diferentes de valuación (y por lo tanto de jerarquización de la calidad académica).
    De todas formas creo que el asunto se hace realmente grave cuando pasa de ser una discusión sobre como medir o no (sobre lo cual pueden haber muchas opiniones y métodos) a cuando los rankings pasan a estar inscritos en otras decisiones. Un caso muy grave en esta dirección es lo que pasa con el sistema de acreditación de carreras en Chile. Que pasó de ser una forma de generar un signo de calidad, a la puerta de acceso a los créditos con Aval del Estado…Lo que cambia todo.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: