Karnataka students' facial data likely to be 'sold or stolen': Educationists urge CM to ban AI-attendance

A group of educationists and parents has urged Karnataka Chief Minister Siddaramaiah to withdraw the state government’s decision to introduce a facial recognition attendance system in schools. They issued a joint statement warning of the exploitation and risks associated with the use of student data.

Expressing concern about the possible misuse of data to morph images of children on an industrial scale, the group asked the chief minister to ban the implementation of the mobile-based Artificial Intelligence (AI)-driven Facial Recognition Attendance System, linked to the Students Achievement Tracking System (SATS).

The signatories include former professors of Delhi University, Jawaharlal Nehru University, Tata Institute of Fundamental Research, and a visiting professor of the National Law School of India University (NLSIU).

Here is the complete joint statement issued by 31 signatories on ‘Ban Facial Recognition Implementation in Schools in Karnataka’.

Ban facial recognition system in schools

1. We note with concern the news that the Karnataka Department of School Education and Literacy has decided to implement a facial recognition system (FRS) from the 2025-26 academic year. Government and aided schools in the state are required to have a mobile-based Artificial Intelligence (AI)-driven Facial Recognition Attendance System, linked to the Students Achievement Tracking System (SATS). The stated aim is to identify regular absentees and provide accurate details on the government scheme beneficiaries at the schools, including midday meals and eggs.

2. We would like to emphasize that FRS have been banned in many geographies for their enormous dangers to people in terms of misuse of the data collected. A major risk from the use of facial recognition in educational institutions is due to a leaky data environment, data outages and even data theft is inevitable. For instance, data captured by CCTV cameras installed by public authorities is routinely shared in the public domain, rendering people vulnerable to shame, loss of dignity, and harm. Recent cases include Caught on CCTV: Madhya Pradesh politician engages in obscene act on expressway, or Women secretly filmed on Bengaluru Metro: Man behind Instagram account arrested. Even nationwide implemented apps, such as one for airport security, permitting quick security clearances for passengers, have had data breaches - Digi Yatra Foundation drops app maker after ‘data-breach.

Also read 18,839 CBSE schools offer AI as a skill subject from Class 6: Education ministry

Risk of data misuse

3. We know that data of students passing SSLC public examinations, collected by the Department of Education, is available to colleges and commercial institutions, which they use to call their families to sell their courses. Likewise, facial data collected by the education department is certainly likely to be ‘sold’ or stolen, and will reach the hands of child traffickers and other criminals.

4. In the context of children, their vulnerability to abuse is much higher, given that they are not emotionally/mentally developed to negotiate crises. Also, the prevailing social norms in India routinely shame victims. For instance, the media routinely reports cases of 'gang rape victims being repeatedly raped, where the criminals threaten to leak videos of these assaults if the victims complain.' There are hundreds of reported cases; these would be less than the proverbial tip of the iceberg.

5. Given that image morphing can convert any images of young girls and boys into potential crime enabling items, any system that captures images of children must be allowed only in exceptional situations, where the need is very clear/strong, and where safeguards can be ensured. Routinely taking photos of children in school settings can cause huge vulnerabilities and harms. This is a real risk because classrooms are not 'public spaces' and are seen as 'protected spaces' where children are free to express themselves, and there will be cases where acts of children (caught on camera, without their understanding) can be used to blackmail them.

Also read Delhi government schools to introduce courses on moral values, AI

Fear of digital blackmail

6. Digital technologies (including AI) are being used to morph images of people on industrial scale, and these are then used for sextortion (blackmail and ransom). AI-powered 'nudify' apps have fuelled a deadly wave of digital blackmail. A "horrific increase" is noticed globally in sextortion cases targeting minors, with victims typically males between the ages of 14 and 17. 15% of high schoolers report hearing about GenAI-generated sexually explicit deepfakes of peers, and 11% of children ages 9-17 know someone who has used GenAI to generate nude images of other children (Laird et al., 2024, Wong, 2024). These challenges extend beyond static images to video and audio, with platforms enabling realistic AI-generated avatars that can mimic real people’s likeness and voices, further complicating authenticity in digital spaces.

7. There is a very large market for videos and images of sexually explicit content, especially of girls. This material is leading to a very large scale of violence and crimes, which is endemic to India, including POCSO cases. We can see reports of crimes, including very violent and gruesome crimes, daily in the newspapers. Only a very small percentage of these would be likely to be reported. Convictions are also abysmally low. This makes exploitation and harm a real danger to society. In his Independence Day address, the hon’ble Chief Minister of Karnataka spoke of the dangers from the digital sphere, and we need to see FRS in this light of actively enabling harm.

Poor digital literacy

8. Given poor levels of critical digital literacy, people, especially the youth, are vulnerable to cybercrime, cyberbullying. Digitizing personal information is likely to harm them severely, as it is likely to be accessed by cyber criminals. India's suicide rates amongst adolescents and youth is the highest in the world, also amongst Indians, suicide rates amongst young people is higher than other age groups. Digital harm is an important cause for these scary figures.

9. Adults (teachers), too, are vulnerable to image morphing, sextortion, and blackmail. Hence, teacher facial recognition should also be prohibited.

10. The facial data captured will be harvested, and used by harvesting entities to try and profile student mental health information. Such profiling can lead to increased manipulation and exploitation of the students and teachers. Already mental health is a serious issue for children, owing to their digital habits, the current generation has been called the ‘anxiety generation’.

Need to ban FRS

11. Hence, The UN Special Rapporteur for the Right to Education has issued a report which calls for a ban on the use of facial recognition technologies in Education, “(j) Refrain from surveillance, whether physical or online, of educational institutions, staff and students and ban facial recognition technologies from such institutions;”.

12. China, which, along with the USA, has the most advanced AI platforms, has planned to curb the use of facial recognition tech in its schools.

Also read We have to connect school children with AI education from this year: Dharmendra Pradhan

Our demand

Given the high probability of severe harm, and that too, to vulnerable children, we demand that the Karnataka Education Department must immediately rescind the decision to implement FRS in schools and introduce a policy banning FRS in educational institutions in the state.

The desire of the Education Department for greater accountability from schools and teachers, can be met in healthier ways - by investing in the school - local community association. All schools are mandated to be managed with the help of the School Development and Monitoring Committee (SDMC).

Accountability in schools is best enforced by capacity building of teachers and parents to support local accountability mechanisms. Digital technologies give a facade of enforcing accountability, but they will fail over time. Techenabled quick fixes will not enable real accountability, instead they can harm.

Capacity building of teachers and parents will not only support greater accountability but also enable healthy decentralized school management, a requirement of all Education Policy.

Precautionary steps in planning EdTech programs

There is a strong hype about digital technologies, to the extent it is believed to be necessarily beneficial. However, research increasingly shows that digital technologies are capable of huge harm, as we have discussed in this note. Hence, the following steps are essential in introducing digital technologies in school education

There is a need to pilot any new technology before large scale implementation. The pilot must be of a long enough duration to identify harms, which in case of technologies usually take more time to become visible, while benefits are quickly visible (or actively projected by their sellers)

Dangers of addiction to digital devices, misogyny and hatred spread through social media, decline in cognitive abilities, causing of anxiety due to social media use are becoming clear. Any digital technology program that seeks to provide devices to children must document its potential vulnerabilities to such dangers.

Digital technologies are fragile and expensive. A rigorous cost-benefit exercise is essential to decide if the proposed project can be made universally available in a cost-effective manner. Many tech projects are bouquet projects, as they are too expensive to be made available universally. Such projects are iniquitous and further stratify the education system.

Proprietary software locks-in the system to the vendor. Upgrades can be difficult, and the system may fail if the company folds up, which can cause not only huge financial losses (BYJU’s closure rendered the 500 crore investment of Govt of AP a waste), but also severe loss in terms of huge invested teacher and student energies and time, which cannot be retrieved.

The Government of India is actively promoting ‘Digital Public Infrastructure’ (DPI) which aligns to FOSS. Only Free and Open-Source Software (FOSS), which can be owned by the public system, must be used in school education.