Workshop Proposal

Proposal as pdf: CSCWworkshop_Algorithms

Computational algorithms have come to play a key role in many human activities: search algorithms structure our search for information online; algorithms in business management systems suggest how employees are best allocated to different tasks and shifts; algorithms used in peer-to-peer platforms enable new types of ad-hoc trade in labor, skills, knowledge and material goods; and algorithms in monitoring systems, such as traffic management, help forecast events and identify future situations that may require intervention.

In the last three years, we have seen fervent public debates [15] and growing scholarly interest [9, 10, 12] in computational algorithms and the ways in which humans rely upon them. ‘Algorithms’ have emerged as a new topic of CSCW research. Yet algorithms have been the bread and butter of computer technology for decades. So, what is new about computational algorithms? And what has changed to inspire this new interest?

To answer these questions, it is important to chart the phenomenon in its diversity, in particular the human response to algorithms; develop adequate, nuanced yet comprehensive, analytic vocabularies; and contribute design implications for both algorithms and the technologies in which they are embodied. This workshop seeks to do this with a focus on computational algorithms in work practices. In the following sections, we review existing research on algorithms and outline the focus, goals and activities of the workshop.

Problems in computational algorithms
Drawing on large amounts of detailed data, computational algorithms are increasingly employed as instruments of prediction, evaluation, and coordination of human behavior. At the same time, algorithmic complexity may generate output that users find difficult to understand and rely upon. Scholars have begun to problematize three major concerns that the increasing proliferation of computational algorithms raises:

First, computational algorithms often ‘hide’ their functioning from their users and bystanders. They are, some argue, to a large degree technically and intellectually inaccessible to most of us: “[…] algorithms remain outside our grasp, and they are designed to be” [4, p. 192; 15; 5]. Given their computational complexity, design, and implementation, computational algorithms can generate impenetrable outcomes, that is, ones which can be difficult for different actors to make sense of. According to a particularly stark claim, not even the developers of some computational algorithms (i.e., the ‘authors’ of their code) understand precisely what computational algorithms do [1]. Sense-making, however, is a pre-condition for trust and, subsequently, for competent and sustained technology use [16].

Second, effective computational algorithms are typically more than data-processing code. They crucially rely upon human work. Computational algorithms often prescribe protocols for human work [7] and thereby make human work the prolonged arm of computation [3]. Insofar as algorithms are more than code, simply rendering ‘hidden’ code technically and intellectually accessible will not do. Instead, algorithms may have to be understood as an iterative socio-technical performance [6].

Third, algorithms are transforming work practices and consumer experiences, but they may not always do so in a fair way [8]. Algorithms are frequently represented as objective, fair, and trustworthy by platform owners and users [4, p. 179; 10; 13]. However, this portrayal of algorithms hides the human labor on which they rely and discourages questioning of the “fair” decisions made by these algorithms. However, algorithms can—intentionally or unintentionally—make discriminatory decisions that may affect users and workers [14, 15]. It is unclear how such discriminatory judgment can be identified because the inner workings of algorithms are often unknown and accountabilities of developers, workers, or the algorithm itself are difficult to determine [1, 11].

Research on algorithms in the workplace
Recent research in CSCW has begun to examine how algorithms change work practices in a variety of different workplaces and emerging domains. In particular, research has explored how algorithmic management influences workers in the “peer economy” and through “microwork”.

The emergence of the peer economy, which uses peer-to-peer platforms to enable new types of ad-hoc trade in labor, skills, knowledge and material goods [2, 8] is just one recent and prominent example of how “algorithmic management” [9] plays an increasing role in the production and consumption of services. Peer-to-peer platforms use algorithms to manage large numbers of typically small interactions between individuals. How the algorithms are constructed (e.g., what they take account of and what they do not) plays a direct role in the experience of the service for both the individual service provider and receiver. Yet it is the platform owners who determine what the algorithms take account of and typically their workings are not revealed to the users.

Research has also explored the ways in which human labor supports algorithmic decision-making. The function of many computational algorithms relies upon human microwork, crowdsourced micro-tasking [7], a division of labor that has been described as “heteromation,” i.e., as enlisting humans for critical tasks [3, 10]. As concepts, microwork and heteromation focus on technology users who have little power over the technological systems they deal with.

To fully account for the presence of computational algorithms in the workplace, future research will need to study not only the microworker but the “macroworker,” the powerful decision-makers who implement algorithms to their advantage as well.

Focus: algorithms at work
This workshop focuses on computational algorithms and their role in the workplace, a domain where human labor and computation are increasingly intertwined. The workshop discusses how data-intensive work practices rely on computational algorithms and how computational algorithms rely on human work—these work practices constitute a new division of labor between collaborating humans and technology. This division of labor is likely to emerge as key characteristic of the future of human work.

To account for such division of labor, we need to characterize:

  • where computational algorithms are used in workplaces (i.e., in what sorts of workplaces and activities),
  • what computational algorithms contribute to work practices (e.g., how they filter, rank, and coordinate human activity),
  • how deeply and how critically this contribution is understood by human collaborators,
  • what human workers contribute to make computational algorithms work,
  • how algorithms influence labor practices at different infrastructural layers (i.e.., from software to hardware),
  • what physical infrastructures are necessary to support algorithms,
  • what human labor is involved in developing and maintaining these infrastructures,
  • what impact working with algorithms has on workers as individuals and collectively,
  • how this impact may serve as a feedback for altering work practices or for designing better algorithms,
  • how these evolving work practices and algorithms impact labor markets; how they create new kinds of human labor and supplant others.

Workshop Goals: Charting Empirical Diversity, Shaping Analytic Vocabularies and Conceptualizing Design Fundamentals
As a first step, the workshop will discuss the presence of computational algorithms in diverse work practices, ranging from automated journalism to employee management, from to the municipal administration and bureaucratic decision-making to the non-traditional working arrangements of crowdsourcing, peer economies that ride-hailing services, and cryptocurrency mining illustrate. As a second step, the workshop will probe different analytic vocabularies that account—in fruitful and critical ways—for such empirical diversity. Finally, given our understandings of how algorithms impact work practices and peoples sense-making activities, we will attempt to conceptualize common themes and implications for the design of algorithms and the technologies through which they are enacted to improve the experiences of workers and other users.

Investigating algorithms at work
In order to understand the contributions that algorithms make to work practices we suggest that researchers explore the similarities and differences across different domains, examining how various actors in these domains make sense of and perceive algorithms. Actors include workers being ‘managed’ by the algorithms (e.g. Uber drivers), customers or clients where algorithms impact on a service (e.g. Uber customers), as well as the people providing the service/platform (e.g. Uber themselves).

Workers engage in sense-making efforts when confronted with proprietary and complex computational algorithms that manage their work practices. For example, algorithms assign passengers to Uber drivers; however, these algorithms do not take into account driver preferences [9]. How do these sense-making efforts help workers discern when they can trust algorithmic judgment? What kinds of strategic workarounds do they develop and utilize?

In some services algorithms output specific recommendations, such as user ratings or recommendations for courses of action. These typically appear as the summation of a variety of individual inputs, e.g., many individuals may rate a specific driver in Ola (an Indian company similar to Uber); or in an investment platform the algorithm may produce a single ‘invest’ recommendation as output from multiple trade analysts. How do users make sense of these recommendations? Do they trust them? Can we add value by representing the diversity or range of inputs, rather than taking the sum of the whole? That is, is it useful to enable users to understand how algorithms reflect specific user perceptions and, if so, what are the ways of doing this?

Lastly, how do platform providers make sense of the complex code involved in their algorithms and how workers and users interact with this code? How does this understanding influence their design decisions?

Algorithms at work–analytic vocabularies
Although algorithms have become an emerging research topic in CSCW and related fields, the word “algorithm” has not been well defined. Researchers discuss a wide range of algorithms, from crowdsourcing algorithms to search algorithms to prediction algorithms. However, while this diversity of algorithms in the workplace has contributed to the richness of this area of research, the lack of conceptual clarity creates difficulty in analyzing these algorithms as a whole. In order to account for this empirical diversity through analytic vocabularies we must examine the following:

  • what we mean when we talk about (computational) algorithms,
  • why we talk about algorithms rather than artefacts, systems, computers, routines, or code,
  • where we draw boundaries to determine what are and are not algorithms,
  • what attributes algorithms have,
  • how we categorize algorithms,
  • which algorithms we choose to discuss and study in our research (and which we do not).

Analytical vocabularies can facilitate integration of the work on algorithms into existing conceptual traditions, which will become increasingly important for contextualizing the research on algorithms. Researchers have pointed to the difficulty in developing empirical studies of algorithms [12], whose inner mechanisms are often hidden. Integrating research on algorithms in the workplace into existing conceptual traditions may give researchers a starting point for developing more robust methodologies for studying diverse algorithms.

We propose possible starting points for a conceptual discussion:

  • algorithms as mediation of human practice, platforms that enable human-to-human interaction,
  • algorithms as performance, a heterogeneous performance involving both human labor and computation,
  • algorithms as infrastructure, a certain kind of structure (often invisible, beyond individual grasp, ready-to-hand), shaping and shaped by human activity; this infrastructure is the automated manifestation of managerial power through computation rather than purely through human judgment.

Design implications/guidelines
Given our understandings of how algorithms impact work practices and sense-making activities, we aim to conceptualize implications for design and identify common themes across domains. Two areas promise to be fruitful areas for design:

  1. How can algorithms be designed for sustainability (of labor markets, of cities etc.) and fairness? For example, to create a sustainable, fair labor market, the concerns of all actors within that labor market should be taken into account. Such actors can include workers, employers, platform owners, and customers. How should these concerns be weighted, balanced and embodied in the algorithms? How should accountability be built into these systems to ensure that actors have recourse when algorithms are unsustainable or unfair?
  2. How can the technologies through which the algorithms take effect be designed to enable sense-making? How can design enable users to understand, and act on the basis of their understanding in a productive way, even where the full complexity of the algorithm remains hidden? How can technologies be designed in ways that are both beneficial for workers and for the whole system?


  1. Solon Barocas, Sophie Hood, Malte Ziewitz. Governing Algorithms: A Provocation Piece. Social Science Research Network, Rochester, NY. Retrieved October 4, 2015 from
  2. David Martin, Benjamin V. Hanrahan, Jacki O’Neill, Neha Gupta. 2014. Being a turker. Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing, ACM, 224-235.
  3. Hamid Ekbia, Bonnie Nardi. 2014. Heteromation and its (dis)contents: The invisible division of labor between humans and machines. First Monday 19, 6, doi: 10.5210/fm.v19i6.5331
  4. Tarleton Gillespie. 2014. The Relevance of Algorithms. In Media Technologies. Essays on Communication, Materiality, and Society, Tarleton Gillespie, Pablo T. Boczkowski, Kirsten A. Foot (Eds.). MIT Press, Cambridge, MA, 167-193.
  5. Kevin Hamilton, Karrie Karahalios, Christian Sandvig, and Motahhare Eslami. 2014. A path to understanding the effects of algorithm awareness. CHI’14 Extended Abstracts on Human Factors in Computing Systems, ACM, 631-642.
  6. Lucas D. Introna. 2013. Epilogue: Performativity and the becoming of sociomaterial assemblages. In Materiality and space: Organizations, artefacts and practices, Francois-Xavier De Vaujany, Nathalie Mitev (Eds.). Palgrave Macmillan Press, Basingstoke, UK, 330-342.
  7. Lilly Irani. 2015. The cultural work of microwork. New Media & Society 17, 5, 720-739.
  8. Tamara Kneese, Alex Rosenblat, danah boyd. 2014. Understanding Fair Labor Practices in a Networked Age. Open Society Foundations’ Future of Work Commissioned Research Papers. doi: 2139/ssrn.2536619
  9. Min Kyung Lee, Daniel Kusbit, Evan Metsky, Laura Dabbish. Working with Machines: The Impact of Algorithmic and Data-Driven Management on Human Workers. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM, 1603-1612.
  10. Caitlin Lustig, Bonnie Nardi. 2015. Algorithmic Authority: The Case of Bitcoin. 2015 48th Hawaii International Conference on System Sciences (HICSS), 743–752. doi: 10.1109/HICSS.2015.95
  11. Adrian Mackenzie. 2006. Cutting Code. Software and Sociality. New York: Peter Lang.
  12. Wanda J. Orlikowski and Susan V. Scott. 2015. The Algorithm and the Crowd: Considering the Materiality of Service Innovation. MIS Quarterly 39, 1, 201–216.
  13. Christian Sandvig. 2014. Seeing the Sort: The Aesthetic and Industrial Defense of “The Algorithm.” Journal of the New Media Caucus 10, 3.
  14. Latonya Sweeney. 2013 Discrimination in online ad delivery. Communications of the ACM, 56, 5, 44–54.
  15. Zeynep Tufekci. 2015. Algorithmic Harms beyond Facebook and Google: Emergent Challenges of Computational Agency. Colorado Technology Law Journal 13, 203-217.
  16. Karl E. Weick. 1990. Technology as Equivoque: Sensemaking in New Technologies. In Technology and Organizations, Paul S. Goodman, Lee S. Sproull et al. (Eds.). Jossey-Bass, San Francisco, 1-44.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s