Disclaimer: This is a shortened and updated version of Section 4 of my article Liable and Sustainable by Design: A Toolbox for a Regulatory Compliant and Sustainable Tech
You can read article in full (in open-access) here:
#digitaleconomy #Web3 #Web4 #websustainability #digitalecosystem #techlaw #innovation #innovationlaw #startups #digital #cybersecurity #compliance #duediligence #data #privacy #dataprivacy #profiling #scoring #humanrights #AI #GDPR #nFADP #DPDP #DigitalPersonalDataProtectionBill #NorraStockholmBygg #CJEU #dataprotection #EUAIAct
Now that I have your attention (food is good, old but gold, right?), let us delve into some tenets of data protection & governance
Profiling is the establishment of a probability concerning the ability of a person to access a certain service. When it is done on the basis of algorithmic decision-making, such establishment is said to be automated, and constitutes scoring. Profiling and scoring by AI today provide for decisions with legal, socio-economic, psychological, and similarly far-reaching effects on most of the data subject’s areas and stages of life, from school to the final years, and even to funerals.
(A necessary introduction that you will see in every recipe :)
The pandemic has exacerbated the effects of the digital transformation: the extractive economy is steadily giving way to the new economic space—the digital economy. This transformation shakes the very foundations of the existence and purpose of law, i.e., the regulation of social relations. However, today, the consequences of developing tech in an unsustainable manner are becoming obvious. Unsustainable tech development contributes to trust erosion, misinformation, and polarization, leading to such legal/ethical issues as irresponsible practices of all sorts, unsafe and insecure digital market, inequality, lack of transparency, breach of privacy, etc.
These developments occur partly because the algorithms that shape our economies, society, and even public discourse were developed with few legal restrictions or commonly held ethical standards. Consequently, ensuring that technologies align with our shared values and existing legal frameworks is crucial.
This series of blog posts explores existing and prospective legal and regulatory frameworks that make tech not only legal by design but also, and especially, liable and sustainable by design. The key questions include whether new laws are necessary or if existing legal, regulatory, and ethical concepts can be adapted (or both!).
It is argued here rather in favour of the adaptation of pre-existing legal concepts, ethical standards, and policy tools to regulate the digital economy effectively, while paying attention to possible gaps and possibilities to fill those gaps. The objective is to synthesize these concepts, analyse their applicability to Web 3.0 and Web 4.0 regulation, and provide a toolkit (see below) for a regulatory compliant and sustainable tech. The blog series focuses on organizations involved in tech and innovation, particularly Web 3-4 actors, using systems analysis to examine regulatory constructions both functionally and institutionally.)
Figure 1: Toolbox. Source: Anna Aseeva (2023), 'Liable and Sustainable by Design: A Toolbox for a Regulatory Compliant and Sustainable Tech', in Sustainability, Vol.16, https://doi.org/10.3390/su16010228
In Episode 3 of the series, I analyse the most relevant existing concepts, recent practices, and avenues in data governance applicable to tech organizations.
Recipe #3: Data Governance
1. In this episode
Unsustainable tech development contributes to trust erosion, misinformation, and polarization, leading to such legal/ethical issues as irresponsible practices of all sorts, unsafe and insecure digital market, inequality, lack of transparency, and, importantly, breach of privacy. These developments occur partly because the algorithms that shape our economies, society, and even public discourse were developed with few legal restrictions or commonly held ethical standards.
It is thus becoming increasingly obvious that the technologies that currently shape socio-economic relations must be consistent with both our shared values and the existing legal and regulatory framework. Take data protection. Confronting the abuse, exploitation, and manipulation of data collected via the internet is a fundamental rights issue (that is, the legal challenges related to these harvested data have implications for privacy and personal safety), as well as the one of contract law (e.g., limits to the freedom to operate) and intellectual property rights (e.g., copyright issues), etc.
In that regard, for example, the European Union’s (EU) General Data Protection Regulation (GDPR) is today considered the world’s strongest data protection law. It regulates how individuals (and now, increasingly, algorithms) access and use personal data and draws boundaries between what organizations and businesses can and cannot do with our personal data. Undoubtedly, the GDPR has introduced privacy by design. Since the main question of this blog series is how to correctly introduce tech that is legal by design, but also, and especially, tech that is liable and sustainable by design, in this blog post I delve into tech liable with data protection laws and governance.
2. Data protection laws around the world
Personal data, as well as a clear information on how exactly an online operator gathers and manages such data, are today governed by increasingly strict regulations around the world (see, e.g., here, here, and here) and are thus quickly becoming sine qua non conditions that any digital business must meet. There is also a growing body of court cases, mostly in the EU, and for the moment, chiefly, but not only, targeting Big Tech firms (such as Amazon, Apple, Facebook (and Meta, more generally), Google, etc.) that makes governance of, as well as protection of internet users’ personal data stricter every day (see, e.g., here, here, and here).
As the GDPR is by far today’s strictest data protection law, any digital business in any way related to the EU single market (such a relationship is quite global for obvious reasons) will have to establish a GDPR register and determine the purposes of data collection; the exact types of collected data; the retention periods; the recipients, including subcontractors; conditions of data transfer outside the EU, etc.
For instance, if an online business, say, a DAO, is registered in Switzerland as a Swiss legal entity (because, as we saw here, unlike most EU members, Switzerland fully recognizes DAOs), it will have to comply with the GDPR if it processes the personal data of individuals located in the EU, offers its services to EU citizens, and/or in any way monitors the behaviour of EU citizens. The GDPR will also apply to any Swiss-incorporated online business, not only to DAOs, if one of these conditions is met, and, in addition, if a company offers any goods or content to EU citizens (that is, does so on the EU market). Note that Switzerland recently adopted its own data protection law. The new Federal Act on Data Protection (nFADP) came into force on 1 September 2023.
The most important difference between GDPR and nFADP is that in the EU, only companies are held liable for the data privacy-related issues described earlier in this section, whereas in Switzerland, any private data operator can now be found liable under nFADP and then fined up to 250,000 CHF.
This particularity of the new Swiss data law may in theory have some consequences for the liability of non-registered DAOs, as discussed here with respect to corporate and contract law. Imagine, for instance, a (obviously, non-Swiss incorporated) DAO that is based in a jurisdiction that does not grant it legal personality. This DAO offers content, goods, or services on the Swiss market, processes the personal data of individuals located in Switzerland, and/or in any way monitors the behaviour of Swiss citizens. Unlike GDPR, nFADP will in theory apply and will likely result in the personal liability of every participant of such a DAO, with possible fines up to 250,000 CHF.
What if a digital business does not collect personal data, but only wallet addresses or transactions? That might actually be the case for many DAOs, as well as of other kinds of the so-called FinTech startups and other types of business organizations, working in one way or another with cryptocurrency. The main scenarios that are possible here are as follows. First, wallet addresses could be considered personal data insofar as they make a person identifiable. Hence, an entity must carry out the compliance measures related to identifying its customers (chiefly, the KYC information that I discussed here). Second, if an entity does not itself do the KYC, the customers may create their wallet address on another platform (custodial wallet). In this case, the concerned online platform has to carry out the compliance measures.
3. AI, online platforms, and personal data
Another crucial point with regard to data protection is the increased gathering, processing, and usage of data by last-generation artificial intelligence (AI), including the general-purpose AI models (such as large language models (LLMs), GPT-4, etc.).
Take, for instance, the so-called legal intelligence platforms proliferating today, especially in Europe. They are supposed to help legal professionals automate a certain portion of their work, especially by (i) doing online legal searches (collection of legal documents) and (ii) creating comprehensive documents (enrichment thereof). These platforms’ AI does these tasks by finding on the web and automatically highlighting crucial information, which is then enriched with relevant external data such as legislation, court rulings, commercial registries, etc. Most typically, one can find both the original and the enriched documents in the database of the platforms. Among these documents are court decisions.
With the publication, enrichment (and eventual re-usage) of court decisions, the main hurdle is an uneasy balance between (i) personal data and the overall privacy of each person mentioned in the decision, on the one hand, and (ii) public policy, and, specifically, the interest of the general public to access the court decisions, on the other (you can find more information and analysis here).
With regard to personal data/privacy, the fundamental rights at stake are:
(i) right to privacy;
(ii) right to personal integrity;
(iii) in the context of ‘sensitive’ litigation, the risk of disturbances or reprisals following the publication of the judgments, particularly for decisions related to such issues as terrorism, family, etc., and hence, the right to avoid / interest in not suffering from these risks; and, finally,
(iv) interest in not suffering the inconvenience of seeing one or more sensitive court decisions freely accessible on the internet (as a derivative of a right to digital/information self-determination).
On the opposite scale (public policy/ interest), at stake are:
(i) right to a fair trial;
(ii) principle of publicness of court decisions;
(iii) right to reuse public information;
(iv) right to information; and
(v) freedom of expression.
First, the principle of pseudonymity, or concealment of personal data and information, postulates that the courts are responsible, according to the practice and/or the regulations applicable to that court, for the initial concealment of information (typically, the surnames and first names of natural persons) before the platforms get these data. In such cases, the platforms receive only an already anonymized version of the decision.
Note that the GDPR provides for data minimization in court proceedings (Article 5(1)(c)). Additionally, the GDPR enshrines a pseudonymization (concealment) measure (Article 4(5)). The latter is limited to certain specified categories of personal data, which is a measure of security (among others) equally provided for in the GDPR (Article 32).
The application of these measures is to be proportionate to the risks associated with the processing. Unlike the GDPR’s obligation to anonymize data, which is an obligation of result, pseudonymization is hence arguably an obligation of conduct (for the legal intelligence platforms, in the context of this analysis).
It is impossible to attain zero risk of breaching privacy in this context. However, if the European online platforms comply with the GDPR and other relevant EU law, including case law, it is very unlikely that there will be a substantive risk for, and significant harm to, or adverse impact on, the concerned individuals.
4. ‘Profiling’ and ‘Scoring’ by AI
Another current issue regarding personal data is profiling. Profiling could be defined as the establishment of a probability concerning the ability of a person to access a certain service. When it is done on the basis of algorithmic decision-making, such establishment is said to be automated, and constitutes scoring. Profiling and scoring by AI today provide for decisions with legal, socio-economic, psychological, and similarly far-reaching effects on most of the data subject’s areas and stages of life, from school to the final years, and even to funerals. Interestingly, both profiling and scoring were defined in the same court case, namely, in SCHUFA Holding.
GDPR Article 22(1), for instance, says that a person has ‘the right not to be subject to’ certain decisions, such as profling and scoring. However, EU law scholars often assume that this right also implies a prohibition (with exceptions) of such decisions (see, e.g., here and here). The CJEU's judgments of December 2023 in the above case and in joined cases C‑26/22 and C‑64/22 confirmed that the two data processing practices by credit information agencies are prohibited in principle by the GDPR.
5. First rules for AI in the world: EU AI Act
Importantly, the recent political agreement on the EU AI Act has been reached on 8 December 2023. The upcoming AI Act is expected, among others, to protect and enhance the rights of EU citizens to file complaints about AI systems and receive explanations of decisions based on high-risk AI systems that significantly impact their fundamental rights, including through the data processing practices discussed just above, as well as similar practices.
As per the previous proposals, the EU AI Act would present an extensive list of prospectively prohibited AI practices, including bans on intrusive and discriminatory uses of AI, such as:
‘real-time’ and ‘post’ remote biometric identification systems in publicly accessible spaces;
biometric categorization systems using sensitive characteristics, such as gender, race, ethnicity, citizenship status, religion, political orientation, etc.;
predictive policing systems based on profiling, location, or past criminal behaviour;
emotion-recognition systems in law enforcement, border management, the workplace, and educational institutions; and
untargeted scraping of facial images from the internet or CCTV footage to create facial recognition databases, in violation of human rights and the right to privacy.
In addition, in the initial discussions (especially, as per EU Parliament) a list of high-risk AI applications was proposed: AI systems that pose significant harm to people’s health, safety, fundamental rights or the environment, or else AI that may influence voters and the outcome of elections and in recommender systems used by social media platforms (with over 45 million users).
Providers of foundation models on the EU market would have to assess and mitigate all possible risks and register their models in the EU database before their release on the single market. Generative AI systems based on such models, such as ChatGPT, would have to comply with transparency requirements (disclosing that the content was AI-generated, also helping distinguish deepfake images from real ones) and ensure safeguards against generating illegal content.
Detailed summaries of the copyrighted data used for their training would also have to be made publicly available under the Act. In my next blog post (# 4), I analyse the most relevant aspects of AI data copyright and other intellectual property-related aspects.
Coming back to the compromise, compared to the text of the initial Commission proposal, the main new elements of the provisional agreement reached on 8 December 2023 are as follows:
specified rules on high-impact general-purpose AI models that can cause systemic risk in the future, as well as on high-risk AI systems
a revised system of governance with some enforcement powers at EU level
extension of the list of prohibitions but with the possibility to use remote biometric identification by law enforcement authorities in public spaces, subject to safeguards
better protection of rights through the obligation for deployers of high-risk AI systems to conduct a fundamental rights impact assessment prior to putting an AI system into use.
Importantly for this blog post on privacy and data protection, the compromise also contains changes clarifying the allocation of responsibilities and roles of the various relevant actors, in particular providers and users of AI systems. It also clarifies the relationship between responsibilities under the AI Act and responsibilities that already exist under other legislation, such as the GDPR and sectorial legislation.
Last but not least, as for some uses of AI risk is deemed unacceptable, along with predictive policing for individuals, biometric categorisation to infer sensitive data (such as sexual orientation or religious beliefs), the untargeted scraping of facial images from the internet, and CCTV footage, all listed above in the earlier AI Act talks, the deal bans scoring discussed in section 4 of this post on profiling and scoring (as well as emotion recognition in the workplace and educational institutions).
6. Summing up
The analysis in this post showed that the governance and regulatory gaps regarding personal data protection are today increasingly made up by strict regulations around the world. There is also a growing body of case law that makes internet users’ personal data governance and protection stricter every day.
At the time of writing, the GDPR—the EU data protection law pre-existing the COVID-related digital revolution—is still the strictest data protection set of rules. Furthermore, the new Swiss data protection law, nFADP, came into effect in September 2023. In some respects, the nFADP is likely to be even stricter than the GDPR: while the latter applies only to companies, the former binds any private data operator that, if found liable, could be fined up to 250,000 CHF.
The last, but definitely not least—and growing—pain point regarding data protection is data’s increased gathering, processing, and usage by last-generation AI, including general-purpose AI systems. Among the key issues here are profiling and scoring. In the EU, the two practices are prohibited in principle by the GDPR, as it was confirmed by the latest CJEU rulings of December 2023.
In the EU, the year 2023 marked several other milestones with the project, and, more recently, a deal on the provisional agreement of the EU AI Act. The Act is expected to further the protection of EU citizens’ right to file complaints about AI systems and receive explanations of decisions based on high-risk AI systems that significantly impact their fundamental rights.
Once finalized, the AI Act will present comprehensive lists of prospectively prohibited AI practices, as well as lists of high-risk AI applications. The prospective lists of prohibitions include social scoring, emotion recognition in the workplace and educational institutions, predictive policing for individuals, biometric categorisation to infer sensitive data, such as sexual orientation or religious beliefs, the untargeted scraping of facial images from the internet, and CCTV footage.
Comments