International Center for Cultural Studies

【ACS Institute 2023 -Day 1- 8/10- Activity Report】Decentering automated decision-making and platformization: challenges posed by algorithmic power

2023-11-14

Plenary Aug 10^th -

Title: Decentering automated decision-making and platformization: challenges posed by algorithmic power

By KWOK Man Yee, Sophie, Master’s student of Inter-Asia Cultural Studies Program, National Tsing Hua University

The plenary consists of three parts, starting with Heather Horst’s remark, followed by the presentations by Cheryl Ruth Soriano and then Adam Sargent. This talk presents their project designed to map the current state of automated decision-making (ADM) as it is unfolding in regions beyond North America and Europe and to develop an understanding how automation may increasingly shape life and industries into the future. It includes the framing of approaches to decentering ADM, aiming to destabilize and displace the commonplace narratives about the origin stories and the futures of AI and ADM.

The project as an “active decentering” is to challenge dominant narratives of directional flows of algorithm power, focusing on the parts of the world that are left out of the global narratives of ADM. Therefore, the team has prioritized the Global South and regions such as the Pacific and the Caribbean which are less visible to demonstrate the diversity of ways in which ADM is the anticipated practice. Acknowledging that there are inequalities in power relationships among the countries in Global North and South, Horst stresses that we should not presume that they would have fixed effects or shapes across the diverse fields of ADM and that histories of colonialism, empire and global political economy would not simply be replayed in digital form.

This first approach is to understand the embeddedness of ADM in the global political economy. Automated decision-making is much depicted as operating outside of the confine of space and time, but place actually tends to reinforce existing norms in the world, just as how technical innovations and their domination start in the West and extend across the globe. It is observed that global power relations are largely emphasized in works on decolonization of the Global South as systems of power that decide the extraction of data and make a profit for enterprises in the Global North.

The geographical mapping of research is important for building up from the work on data colonialism and the broader political ADM. It is equally important to look at the networks of funding, yet it is not just about the direction of money flow, but also the kinds of support and infrastructure that are established, for example, the mapping of corporate global research centers. The result of the initial mapping of research locations shows that ADM systems travel across the globe in a highly uneven way.

A key dynamic of ADM is observed in Latin America – there are not many investments in research and development in AI by large global tech firms in the region, but if there are, they are mostly developed through local startups and infrastructures. On the other hand, ADM systems in South Asia and Africa are being developed with direct or indirect support from large Tech companies, primarily based in the US. In addition to corporations, national narratives and state-level planning serve as important in shaping the possibilities for AI automation and ADM practice, as well as controlling digital economy. In some countries, digital economy is treated as the next development of industry and repeats earlier development narratives. Yet, different countries have different takes on their positioning.

The second approach focuses on the diversity of automated decision-making in practice and starts from the heterogeneity of existing practices which generate, modify, and reproduce systems of ADM. Practices shape the ways of assessing technical systems. This project’s focus is on capturing the practices that people in the world are engaged in and suggests how these practices may rely on and create different understandings of what technologies like machine learning models, drones, and chatbots can do.

Horst demonstrates in the introductory remarks the need to look at how ADM is used in diverse social, historical, political, as well as economic contexts from the emergence of platform labor arrangements to digital contract work. The project aims also to provide multiple accounts of the political economy of ADM automation and AI and to understand the diverse goals and trajectories of ADM.

The second part of the plenary is a presentation that focuses on the Philippines, by Cheryl Soriano, Professor of Communication at De LaSalle University, highlighting the transforming relationships between platform workers and platforms, clients, and others on digital platforms. It is known that the subjugation of labor to algorithm management systems contributes to the fragmentation of work which undermines the standard deployment relationship, and even possibly the potential for solidarity. Soriano examines the digitality in platform-mediated organization work, advances a critique of labor platformization, and places emphasis on the contradictory experiences of precarity that platform laborers have which leads to the emergence of agency and solidarity.

To begin with, Soriano emphasizes how the post-colonial condition of countries like the Philippines matters to the agency of platform laborers. The Philippines has a reputation as a transnational service workforce powerhouse, formed through its historical trajectory of post-coloniality and the context of labor, which further extends to the present neoliberal global order. Filipinos work as online freelance workers such as online secretaries and virtual assistants, partly because of the government’s strategy of making platform labor a solution to unemployment or underemployment, bringing in foreign currency to the country. Like other large labor suppliers of online freelance work, the Philippines have predominantly foreign clients, yet the rates workers receive are low. This implies that the Philippines is in a subordinate position that provides cheap labor in the global economy. However, given that precarity has been an existing condition for Filipino workers, cloud work is perceived as a viable and better work option compared to overseas workers because it allows workers to earn Dollars without having to be away from home, and to perform as a global worker.

Moving on, Soriano talks about the discourses of agency and relational labor practices around labor outsourcing, with contrasting social and economic spheres. Her findings show that the relationships between workers and clients in cloud work are often marked by cultural and power distance beyond the physical distance, as well as varying time zones. To bridge these distances, technology is the mediator. Cloud workers are also in a highly insecure position due to fierce global competition and labor seasonality, which lead to their willingness to accept more tasks than previously agreed or deliver a task faster than necessary, without asking for compensation. Workers see this as being reciprocal rather than deferential, where they could gain clients’ trust and display their competence and efficiency. These are all entangled in a web of colonial and technologically mediated power relations, and workers act according to their imaginary of moral relationships with clients. This leads to the issue of unfair caring practices, which are performed regularly in this work dynamic. Workers find such relational labor care crucial in building the foundation for trust, and in shaping the ways to negotiate the paid aspects of their work. However, beyond pure economic calculations, some workers see mutual disclosure of personal circumstances or problems as establishing a humane relationship with clients where they express concern for one another’s personal lives. Clients also reciprocate, in the form of care or cash advances and material gifts. Hence, such communicative relationships might also be seen as a dynamic one with social and cultural dimensions of care. Facing challenges like no pay or lack of benefits and safety nets, labor seasonality, and lack of career progression, those who can navigate relationships with clients and work around the relational practices are likely to lessen the impact of these challenges.

Worker-to-worker relations are also of importance. In some instances when workers take on large projects, they start to outsource work to other workers, especially to their family or personal network. Pulling workers together, they can now go for bigger jobs or projects and sustain clients on the platforms. Workers also build entrepreneurial solidarity to affirm the vulnerability of platform work through their showcasing of successes in navigating and negotiating relationships with clients, as they start to emerge as coaches, agencies, and influencers on social media, offering help and trying to develop better working conditions. Yet, at the same time, they are also reinforcing the different statuses among themselves.

These all demonstrate the calculations and negotiations that take place among actors in platform economy beyond algorithmic power of the platforms.

The third part of the plenary is by Dr. Adam Sargent, focusing on the increasing use of ADM technologies in the provision of automated loans in India. Digital lending challenges older narratives about artificial intelligence and technological development because sites of such innovation are largely in the Global South rather than the traditional financial centers of the global North.

The predictive power of algorithms capable of analyzing patterns in the personal data of borrowers is the key. Seeking to create new sources of financial accumulation, the un- or under-banked populations are the targets, and so how their data is used is of interest here. Sargent presents his reading on unregulated lending apps and their weaponization of borrower data and goes beyond to see the emerging politics of digital lending.

Fake digital loan providers pose as legitimate lenders through apps that are easily available on Google Play Store. Not much different from other lending providers, these loans come with high interest rates and processing fees, together with strict penalties for late repayment. However, once borrowers are late for repayments, these providers start to bombard them with calls and messages, often with verbal abuse and threats. The differences between fake and legitimate lenders lay in how they use the personal data scraped from borrowers’ phones, including social media usage, phone logs, contact lists, etc.

Digital lending has been promoted by the Indian government as a tool of financial inclusion, and whether legitimate or fake, it highlights how technologies of automated decision-making become entangled with older relational hierarchies of gender, kinship, and social solidarity. With their state support, India saw a boost in the uptake of digital payment technologies, which provides the context for making personal data requests seem less significant. Some digital lenders create a score called a Social Loan Quotient, to predict an individual’s character and willingness to pay, based on smartphone metadata. This, in turn, enables the use of data as collateral as the everyday activities of the borrower are extracted and circulated.

The tactics that collection agents deployed are mostly threats of abuse, through abusive language that brings in patriarchal power relations to debt relations, and even sexualized violence threats by male agents. As shown in an example, dealing with non-payment, the illegitimate agent starts with rather a formal way of speaking but quickly turns to threatening wordings that contain unwanted intimacy, then mobilizes the borrower’s data, claiming that they have her parents’ and friends’ numbers. Sargent points out an important point that because contact lists include connections close and far, the agencies could spread messages indiscriminately, and attempt to mobilize these connections for repayment. Informal lending between kin and friends might also happen in order to repay legitimate loans, and this is reconfigured around a new terrain of contestation. On the other hand, borrowers are also able to manipulate loans in ways that digital lenders cannot control, for example using another person’s phone to access loans due to a gendered scoring system. Relations of kinship, friendship and solidarity then come into play. This also means that loan apps with algorithmic credit scoring systems become entangled in the social relations of interdependence – those that they mobilize and those that are mobilized to subvert them. The hybrid forms of inequality emerge here as social ties are involved, with the inequalities of gender, caste and class drawn into and against digital debt relations.