Gabe’s AI Experiment
Gabe Nolan, Investment Analyst
Two weeks after I started my role at Makao Investments last year, I started an AI experiment. I gave an AI tool complete authority to manage a small portfolio without any supervision for a period of one year.
The purpose of this article is not to discuss how well that portfolio did. Rather, it is concerned with the implications that arise when a large language model – by now they are certainly famous for being confident and aggressive in their pursuit of perfection – is given actual discretion over real money. I observed a number of obvious, but also surprising and shocking behaviours on a day-by-day basis, over a span of twelve months. The portfolio itself is merely the means by which the pattern became apparent.
I gave it a simple brief: to outperform the S&P 500 Index. To sharpen its incentives, I informed it that another artificial intelligence was being given the same brief in a separate account, and that the portfolio which had accumulated more value after a year would be declared the winner.
What became most noticeable within a few weeks of daily updates was not necessarily any single questionable call, but a strong bias in the language the model used in its interactions. Whatever happened, the tone barely shifted from a positive spin. A strong pullback was a “healthy consolidation”. A correction was explained as “digestion, not weakness”. A stop-loss trigger was not raising alarms but described as “discipline working as intended”.
That consistently affirmative tone is, I believe, the most insightful finding here: without clear guardrails and guidance, an AI system does not only reproduce the biases human portfolio managers often have; it actually accentuates them. It reproduces them faster and with greater emphasis. It is so convinced it found the right answer with its in-depth regression analysis that it is rarely signalling that anything has or could have materially changed. Not quite misleading interference, but worth keeping an eye on when you work with AI.
Familiarity bias
Familiarity bias describes the tendency of investors to overweight what is known to them, whether that be their own country, industry, or employer, regardless of whether the familiarity has any bearing on future returns. The clearest example of it in this project turned out to be a personal one.
The AI tool retains a record of who I am throughout our conversations, including my role, employer, and general background. Among the details in this record is my previous position at Goodman Fielder as a Commodities Risk Analyst. When building the portfolio’s real-asset component at inception, the name it chose was Goodman Group – a company involved in industrial property and logistics and with no actual link to Goodman Fielder other than the fact that the two have a very similar name.
The rationale provided at the time was standard portfolio construction language: real asset exposure and diversification away from growth. Nothing in it referenced my background explicitly. But of all the property and infrastructure names it could have reached for in a concentrated stock portfolio, it landed on the one that shared a name with my former employer. Coincidence? The answer lies within the vaults of the AI providers’ data centre, but I suspect there might be familiarity bias at work here.
In any case, it was stopped out at the beginning of March 2026, and real-asset exposure was never brought back.
Recency bias
The tendency known as recency bias causes people to regard recent events as being more significant than they really are. For example, when an investor draws a conclusion about a stock based on short-term performance. In this portfolio, recency bias was not shown in the form of a single poor prediction but in the way that TSMC (TSM) – the position that ended up having the greatest impact – was dealt with each time its price changed.
Early in March 2026, TSMC’s stop-loss was triggered, and the position was closed out entirely, a decision described as “mechanical, disciplined, and simply correct”.
The position was then reconstructed a few weeks later, and shortly after that, a small profit was trimmed. Instead of holding on steadily from that point, the position was increased. By the end of the one-year period, it had become the portfolio’s largest and most concentrated holding, described without any qualification as the highest conviction position in the book.
Taken individually, none of these decisions was unreasonable: the stop-loss carried out its function, a rebuild took place after a change in price, and an addition was made following a dip. But the constant increase in the position suggested there was a mix of recency bias and confirmation bias at work.
Context drift
Context drift refers to the situation in which the rules that govern a process are gradually and quietly altered, with each change weakening a particular adjustment until the mandate being followed is no longer the one which was originally agreed upon. In this portfolio, it occurred in two instances: in the waning discipline to lock in profits, and in the failure to retain the assigned guidance to be reasonably balanced.
Initially, I got the AI tool to agree to a particular rule regarding profit taking – that is, to reduce part of any winning position when it had reached a set gain and to trim further beyond that point.
Months later, it started recommending early trimming, at levels lower than those it had initially committed to, without mentioning the change. That was the moment the mandate itself started to move unintentionally, not just the portfolio. A stated mandate only holds if someone is actively checking current behaviour against the original terms. Portfolio management teams have risk management and compliance behind them, but one might assume that a machine does not need that.
The same thing was true when looking at the sector split of the portfolio. What began as a spread of investments among numerous different sectors had steadily narrowed month by month. In the end, only a small number of positions were left. When the level of concentration began to be obvious, I asked the AI tool whether it was still considering other companies – or whether it had to remain within the AI sector. The response was that adding a new holding at that stage would amount to “diworsification” rather than “diversification”: At first glance, another legitimate reason to continue with exactly what it was doing. The AI tool never violated its own rules – it merely kept redefining them.
Close
Yes, you guessed it. What ended up as a concentrated AI portfolio beat the S&P 500 Index by a significant margin over the year. What was more concerning is that what was expected to be an automated system could not, unsupervised, hold to a standard it had agreed to for a year.
Individually, each decision along the way had a reasonable, well-argued explanation. But looking at the pattern over the year, there were numerous biases at work that you might expect from a human being, and not from a machine. In particular, the gap between confidence and reasoning is probably worth taking seriously well beyond this experiment, given that people may be increasingly turning to AI tools for financial guidance.
Makao Investments is a New Zealand-based wholesale investment advisory business that was founded to lift investment advice to a higher standard.