Risks and limitations

Plagiarism and authorship

Utrecht University’s position on research integrity is described in the UU Code of Conduct for Scrupulous Academic Practice and Integrity and the Netherlands Code of Conduct for Research integrity (2018). In the latter, plagiarism is defined as “the use of another person’s ideas, work methods, results or texts without appropriate acknowledgement”. Whether and to what extent AI-generated text counts as plagiarism is currently still a grey area. It therefore remains your responsibility to ensure that submitted work reflects your own effort, also to reflect rules on authorship described in the Netherlands Code of Conduct for Research integrity.

Originality

When your work is expected to be original, it's important that the audience can reasonably assume that you created the content yourself. This expectation is lower in certain cases, like writing a standard instruction manual or a user guide. Even in those situations, if AI is used to help create the content, it’s still important for a person to review it to make sure it’s accurate.

Transparency 

If you use GenAI in your workflow or research in a ‘substantial’ way (for example, use that involves more than basic text editing support), be transparent about your use of GenAI tools. Although 'substantial' as a term cannot be simply defined, the EU Living Guidelines on the Use of Generative AI in Research provide a starting point to consider this.   

As research support staff, you may sometimes work with material (for example text) that was written by a researcher to whom you are providing support. If you enter their work in a GenAI tool, make sure to ask whether the researcher you are supporting feels comfortable with you using a GenAI tool and specify which tool you intend to use (and how). 

As a researcher, appropriately and transparently acknowledge the use of the source/platform as you would any other piece of evidence/material in your submission. This will vary widely depending on how individuals may use AI tools in specific instances and/or the conventions in different disciplines. More information on citing GenAI in your work. 

Tips on Research Integrity

1. Ensure that any work you submit remains a reflection of your own effort. 
2. In case of substantial use, be transparent (inside and outside the organisation) about your use of GenAI. 

Using GenAI involves processing large amounts of data, which can in many cases contain personal data. Depending on the AI tool you use, the data with which you prompt the model could eventually be used by the model developers to train and improve the models on which the tool is based. As a result, that data can ‘leak’, e.g. be reproduced as output for another user and become publicly available even though that was not the intention beforehand.

Anything done with personal data with the use of any computing tools, including AI, is subject to strict regulations, and the legal definition of personal data is much broader than the lay understanding of the term. For further information, please consult the intranet pages on personal data and privacy. The General Data Protection Regulation (GDPR, or AVG in Dutch) sets rules for handling personal data. If you use GenAI with personal data, you must follow these rules. In academic research, there are some exceptions, but they don’t remove most of the key requirements.  

Before you input data into any AI tool, consult a relevant officer of your department if the data you intend to input into the AI tool is personal data and what conditions apply: every organisational unit in faculties and corporate offices of UU has experts on data governance and protection who can help you. See the intranet page Privacy Officers for more information.

Tips on protecting privacy and data security

1. When using GenAI, treat the information you enter as if you were posting it on a public site (e.g., a social network or a public blog). 
2. Do not use any personal or sensitive data as input when using GenAI tools, unless you are certain that the data protection law is respected. 
3. Many risks pertaining to GenAI are not immediately visible. In many cases, it is necessary to assess the impact of your use of GenAI on data protection and privacy. You are not on your own: UU has experts who are available to assist and advise you with this process.
4. Use UU AI-Chat or private, locally run models on your own computer if possible: this ensures that sensitive and proprietary data never leaves the protected UU environment.

When using GenAI tools, it is crucial to be aware of potential IP issues. In the context of any AI-generated content, questions may arise about who owns the IP rights of the generated content. Current laws do not provide clear answers to AI-related IP issues yet. In case of substantial use of GenAI (see 'Research Integrity'), keep a track record of your workflow (e.g. earlier text drafts and revisions, literature summaries, etc.) to be able to demonstrate that your work is original if needed (e.g. if there is doubt as to the origin of the work).

Similar to personal data, works of others entered into an AI tool can also accidentally ‘leak’. The works of others you input into an AI tool are protected by copyright of the authors and/ or publishers. Inputting those works into AI tools may constitute a violation of copyright. Read the licences under which those works are published. While older licenses do not explicitly account for use of works in AI tools, they will still contain relevant general provisions, and more recent licenses may already contain AI-specific clauses.

Tips on avoiding legal risks

Ensure that the training data of your GenAI model does not infringe on any third-party IP rights and until clear laws are in place, err on the side of caution. To mitigate these risks:

1. When used ‘substantially’ (see 'Research Integrity') in for example publications or grant proposals, make sure to be transparent about the way GenAI was used.
2. Keep a track record of interactions with GenAI.
3. Carefully select your GenAI tool, taking into account IP and data security risks (see also Effective use of GenAI). When possible, use UU AI-Chat or private, locally run models on your own computer: this ensures that sensitive and proprietary data never leaves the protected UU environment.
4. Some tools let you opt out of using your data for model training (under user settings). Others let you opt in and don’t use your data for model training unless you've provided explicit consent. When possible, opt out of model training or use tools that don’t use input data for model training. 

GenAI models are only as good as the data they are trained on. If the training data is biased or incomplete, the GenAI's output will also be biased or incomplete. For example, if a model is only trained on text coming from western countries, it will include the wording, cultural biases, and ideas prevalent in these countries. If, for instance, you happen to be writing a proposal about mental health conditions that are more prevalent in non-western countries, this could be an issue. It is crucial to use GenAI tools that use diverse and representative training data, and/or to screen the output of such tools carefully for potential biases.

Tips on avoiding bias

1. Understand the GenAI’s training data as much as possible. Diverse and representative data lead to fairer outputs.
2. Stay critical and continuously assess the GenAI’s output for biased patterns.

GenAI recognises patterns and produces statistically likely output without understanding the content the way a human does. It can therefore generate output that seems plausible but is factually inaccurate or even nonsensical - a phenomenon often referred to as "hallucination".  The AI might for example confidently describe a non-existent psychological theory called "Cognitive Resonance Theory" and attribute it to a fictional psychologist. Or, when asked for references, it might cite publications that do not actually exist. Since GenAI models seldom express uncertainty, they will tend to state such things baldly as facts, making it harder to detect inaccuracies.

The output of a GenAI model can also vary substantially depending on the way you phrase your prompt, instruction, or question. Thus, effective use may sometimes require modifying a prompt to obtain more relevant or precise output. Note that since these models are probabilistic, the exact same output will never be obtained, even with the same prompt, over multiple queries.

Tips on fact checking GenAI outputs

1. Finetune your prompts to obtain relevant and accurate output and keep a record of your prompts.
2. Always critically evaluate GenAI outputs and check for hallucinations.

While GenAI can speed up processes and help generate novel ideas, over-reliance can lead to a lack of oversight and critical thinking. Some GenAI tools are marketed as time saving ‘AI Research Assistants’ (e.g., Elicit, Scite, Scholarcy). If you choose to use them, use them with caution and be aware of their limitations. Be the human-in-the-loop (or even better; in the driver’s seat) at all times and evaluate the accuracy in the output of the AI.

Tips on maintaining balance with GenAI

1. Use (and promote) a balanced approach, keeping in mind and emphasizing that GenAI is a tool to complement human expertise rather than replace it entirely.
2. Form your own GenAI peer review community: for longer and/or more important output, apply the many pairs of eyes principles that you would also apply for scientific publications.

Running, and especially training, GenAI models require significant computational power, leading to high energy consumption and a substantial carbon footprint. Electricity consumption from data centers is estimated to amount to about 1-2% of the total global electricity consumption and is expected to double by 2030 [1]. Aside from energy use, GenAI indirectly requires large amounts of fresh water for server cooling. When considering the larger picture, the global demand for AI could result in 4.2 to 6.6 billion cubic meters of water withdrawal by 2027 [2]. This figure is greater than the annual water withdrawal of 4 to 6 times that of Denmark or half of the United Kingdom.

Though using GenAI can have many benefits for your work, it is important to balance maximising the benefits of AI, while taking the environmental impact into account. Learning where GenAI adds most value in your specific workflow and being well informed about its resource consumption can help to find a middle ground between these conflicting values.

[1] https://www.iea.org/reports/energy-and-ai
[2] Li, P., Yang, J., Islam, M. A., & Ren, S. (2023). Making ai less” thirsty”: Uncovering and addressing the secret water footprint of ai models.

Tips on reducing your environmental impact

1. Ask yourself whether GenAI is really needed for the task, or whether a less energy-intensive solution (like a browser search or -in a research context- standard machine learning approach) would be sufficient or even better suited.
2. Large, general-purpose models are powerful, but also use a lot of energy. Use smaller, task-specific models when possible for simple tasks (e.g., grammar checks or translation).