In a first anywhere in the world, a court in the Netherlands recently stopped a digital identification scheme for reasons of exclusion
Impact of the move
This has a context for similar artificial intelligence systems worldwide, especially at a time when identity, citizenship and privacy are pertinent questions in India.
Why did the court rule against it?
- The Dutch district court ruled against an identification mechanism called SyRI (System Risk Indicator), because of data privacy and human rights concerns.
- While The Hague district court found using new technology to control fraud was acceptable, it held that SyRI was too invasive and violative of the privacy guarantees given by European Human Rights Law as well as the EU’s General Data Protection Regulation.
- The court found that opaque algorithmic decision-making puts citizens at a disadvantage to challenge the resulting risk scores
About the digital identification scheme-SyRI(System Risk Indicator)
- The Dutch Ministry of Social Affairs developed SyRI in 2014 to weed out those who are most likely to commit fraud and receive government benefits.
- A Dutch law allowed government agencies to share 17 categories of data about welfare recipients such as taxes, land registries, employment records, and vehicle registrations with a private company.
- The tool is armed with big data analytics and uses individuals’ private data on consumption and other significant activities to rank them and create risk profiles that authorities can use in the detection of fraud.
- For instance, SyRI allegedly helped Dutch authorities spy on people in poor neighborhoods only based on algorithmic evidence and no other proof.
