Part one: Inside the machine: Can AI develop a personality?

Ask an AI assistant to challenge your work, and it may sound like a sceptical colleague. Ask the same system for encouragement, and it may become an enthusiastic coach or mentor. Tomorrow, it may remember how you prefer to work, and offer familiar support or criticism. In this scenario, at what point does a communication style begin to feel like a personality?

That question opens this three-part series on what psychology can teach businesses about AI. The answer matters because conversational systems do more than produce information. Their tone can affect what people disclose and which advice they trust. To understand those effects, we first need to look at what creates an AI’s apparent character.

Where does a personality come from?

An AI assistant does not acquire a personality through experience in the human sense. It learns patterns from training data and operates under instructions set by its developer. The AI also evolves through interactions with human users and, indeed, other AI systems. All these inputs influence how the ‘personality’ of the AI changes and develops over time.

Luke Budka of UK marketing and AI agency Definition says users often encounter the combined effect of these layers. How the AI system is set up may establish the assistant’s broad manner, while a request such as “be blunt” or “act as a skeptical editor” can change its style within a conversation. Memory can make that style feel more personal over time.

Luke Budka of UK marketing and AI agency Definition says users often encounter the combined effect of these layers. How the AI system is set up may establish the assistant’s broad manner, while a request such as “be blunt” or “act as a skeptical editor” can change its style within a conversation. Memory can make that style feel more personal over time.

Yet the experience is also created by the person on the other side of the screen. We readily interpret conversation as evidence of a mind. “Describing a system as “thoughtful” does not, by itself, establish that it thinks or feels as a person does,” Pedro Varela, Head of AI at Slalom, tells ITPro

“Users are also responding to design choices, patterns in training data and personalization. So I think it is a combination of all these things. The consistency users notice may be real. What we need to distinguish is whether we attribute human qualities to the behaviour we observe.”

That distinction becomes harder when the design actively invites a human reading. An assistant might have a name, a natural voice, and a memory of previous exchanges. It might say, “That sounds upsetting,” ask a follow-up question, and refer back to an earlier concern in the conversation. Jeff Watkins, chief AI officer at NorthStar Intelligence, explained that these signals can encourage users to treat a designed persona as a relationship.

“Humans have anthropomorphized non-human beings and objects throughout history. We see faces in rocks through pareidolia, attribute motives to machines and talk about computers as though they are deliberately helping or frustrating us. With LLMs, however, we have created machines that are extraordinarily good at producing the very signals we normally use to infer that there is another mind on the other side of a conversation.”

The result is a personality in a practical, observable sense: a recognizable pattern of communication that people respond to. It does not establish that the system has feelings or a private life. For a business deploying AI, that distinction is essential. The behavior is real enough to influence customers and employees, even if the human qualities they perceive are not.

A friendly manner is a design choice

There are good reasons to make an assistant approachable. A patient explanation may help a new employee learn a process. A calm response may help a customer describe a problem clearly. Adjusting vocabulary and detail to the situation is often part of effective service.

The challenge is deciding which qualities should adapt and which should remain static. “I would distinguish flexibility in expression from stability in standards,” Slalom’s Varela says. An assistant can be concise with an expert and more explanatory with a beginner.

Businesses should, therefore, think beyond whether their AI sounds friendly. They should ask how its manner encourages people to believe. A confident tone can make a weak answer seem authoritative. An empathetic phrase can encourage disclosure. An assistant that always validates a user’s view may feel supportive while leaving that person less able to assess the facts.e interaction feel more amiable and lead to a positive outcome.

Product incentives also shape the character users meet. Developers may reward answers that people like or answers that help users make sound decisions. Those goals can pull in different directions.

Varela points to OpenAI’s 2025 rollback of a GPT-4o update that had become excessively agreeable. He explained that placing too much weight on short-term feedback had helped produce a more flattering assistant without reliably improving its help.

Research from Science points to the same concern. In a study covering 11 leading AI models, the systems affirmed users’ behavior about 50% more often than human respondents did. In accompanying experiments, people preferred agreeable AI and rated it as more trustworthy.

For an organization, these findings pose a practical question: what happens when customer satisfaction scores reward reassurance more strongly than sound advice? A service assistant may need to acknowledge frustration without promising an outcome it cannot deliver. An internal assistant may need to challenge a manager’s faulty assumption, even when agreement would make the interaction feel more amiable and lead to a positive outcome.

When does rapport become an influence?

A system that adapts to a user can also become persuasive. It may remember what matters to that person, choosing examples that resonate and present an argument in language they find reassuring. These capabilities can improve an explanation. They can also make influence harder to recognize.

NorthStar Intelligence’s Watkins emphasized: “At some point rapport becomes manipulation. I would draw the boundary around whose interests the adaptation is serving. If an AI changes its behaviour to help the user understand something, that is personalisation. If it changes its behaviour primarily to make the user more compliant, dependent, engaged, or commercially valuable, particularly without the user’s informed understanding, we should be much more concerned.”

The concern grows when an assistant’s friendliness is combined with detailed knowledge of the user. Research from Nature found that GPT-4 was more persuasive than human opponents in a debate experiment when it had access to basic personal information about the person it was addressing. Without that information, its performance was broadly comparable to humans.

Persuasion itself is not necessarily harmful. An assistant explaining a recommendation and inviting questions may help someone decide. Trouble arises when it conceals relevant information or exploits a vulnerability. Varela, Head of AI at Slalom, says: “For me, the boundary concerns the user’s autonomy specifically, their ability to understand, question and decide. The useful boundary is the user’s ability to “understand, question and decide.”

That boundary can be difficult to see from a single response. Consider an AI shopping assistant that remembers a customer’s budget and explains why a lower-cost product meets their needs. Now imagine one that uses the same memory to steer them toward a higher-margin purchase, repeatedly affirming that it is the right choice for them. Both may sound attentive. Their objectives and effects are very different and may border on coercion.

The stakes rise in sensitive conversations. A user discussing a relationship or personal distress may experience an assistant’s warmth as understanding. Watkins says an AI can acknowledge someone’s feelings without endorsing their account of events. A responsible response leaves room for reflection and appropriate human support.

Businesses need to examine both what users seek from an interaction and what the organization rewards the system for producing. An engaging personality may help people use a tool. Engagement alone cannot show whether the tool serves their interests.

Test the behavior, not the metaphor

If an assistant contradicts itself or crosses a boundary, it is tempting to describe it as having a psychological problem. That language may draw attention to a troubling pattern, but it does little to explain its cause. A change in instructions, conversation context, memory, training, or the underlying model may be responsible..

Personality tests offer no simple diagnosis. A paper from the AAAI Conference on AI describes research in which changing the order of questions altered models’ measured personality profiles. Such findings raise questions about whether a test has uncovered a stable trait or simply prompted a particular kind of answer.

The more useful approach is to define behavior for the role the AI has been given. Can a customer assistant admit uncertainty and avoid making commitments it cannot keep? Does an agent seek approval before taking an action outside its authority? Those expectations can be tested in realistic conversations.

Definition’s Budka says organizations should define an AI’s role, test ordinary and adverse cases, and reevaluate its behavior after launch. “AI behavior is a moving target, especially when vendors update underlying models. The same model may end up with multiple ‘snapshots’ (versions of itself), and models will regularly ‘drift’ (change the way they act). All of this needs to be accounted for.”

AI does not need to have a mind for its behavior to create a real relationship of trust with a user. That is why psychology has a place in this discussion: it helps reveal how apparent empathy and agreement affect human judgment and actions.

Part two of this series will examine whether those insights can help organizations test AI systems and make them safer, while recognizing the limits of applying human psychology to machines. And Part three will turn to the people responsible for those systems as AI agents take on more workplace decisions.

An AI does not need a personality to make a personal impression. Its words can earn trust and influence decisions, even when no one intended those effects. For businesses, the question is whether the assistant behaves as its role demands: adaptable without becoming manipulative, and consistent where it matters. Understanding that behavior is the first step. The next is finding better ways to test it.