Social and Ethical Risks Posed by General-Purpose LLMs for Settling Newcomers in Canada
It addresses risks for newcomers and service providers in the Canadian settlement sector, warning against unguarded AI use and calling for customized solutions, but is incremental as it builds on existing concerns about AI ethics in social services.
The paper examines the social and ethical risks of using general-purpose LLMs like ChatGPT for settling newcomers in Canada, highlighting potential detrimental implications for immigrants and refugees due to lack of domain-specific tailoring.
The non-profit settlement sector in Canada supports newcomers in achieving successful integration. This sector faces increasing operational pressures amidst rising immigration targets, which highlights a need for enhanced efficiency and innovation, potentially through reliable AI solutions. The ad-hoc use of general-purpose generative AI, such as ChatGPT, might become a common practice among newcomers and service providers to address this need. However, these tools are not tailored for the settlement domain and can have detrimental implications for immigrants and refugees. We explore the risks that these tools might pose on newcomers to first, warn against the unguarded use of generative AI, and second, to incentivize further research and development in creating AI literacy programs as well as customized LLMs that are aligned with the preferences of the impacted communities. Crucially, such technologies should be designed to integrate seamlessly into the existing workflow of the settlement sector, ensuring human oversight, trustworthiness, and accountability.