I asked ChatGPT for tax help—experts say I fell into a classic trap

I’ve enjoyed a relatively simple tax situation for most of my adult life. Most years, I just made sure my income from my W-2 was correct and clicked through to the questions in my preferred tax software. I have no dependents, no extra income, no property in my name.

Last year was a little different. After years of purchasing shares through my company’s employee stock purchase plan, I sold most of my shares to begin raising money for my upcoming wedding.

There are some relatively tough rules regarding selling these shares, but the gist is that these plans allow employees to purchase shares at a discount to the actual share price. So determining how much money you gained (in which case you owe capital gains tax) or lost on the sale of your shares requires some calculations.

So I did what 1 in 5 taxpayers do these days. According to a recent survey from IPX 1031: I asked AI for help.

I did this with suspicion. I’d seen enough stories about AI “hallucinations” (the industry term for chatbots getting things wrong) that I half-expected ChatGPT to ruin my taxes. And it’s only been three years since we put AI to the test on tax strategies and watched it flounder. It is also worth noting that OpenAI’s usage policies Be careful not to use its product to automate “high-risk decisions without human review in sensitive areas.”

However, once I started chatting with the latest version of OpenAI’s large language model, I could feel my hesitations disappearing. Not only did he answer my initial question about how ESPP sales are taxed, but he also broke things down into digestible items and asked me if I was comfortable sharing more information.

Because I use an enterprise version of the software that does not use data to train OpenAI models, I uploaded the consolidated 1099 form from my brokerage.

“This is great – [your brokerage] “He actually gave us everything we needed,” the bot told me. “Here’s what happened.”

What ChatGPT told me is this in a nutshell: Your broker is using a number loaded into your tax software. But you actually need to use a different trick. I had to check my last few W-2s to see if they contained a specific line item.

I was ready to press “file.” But first, I met with certified public accountant Miklos Ringbauer, who had helped me with tax stories before.

He told me that I was probably getting accurate but also incomplete information. The W-2 check ChatGPT assured me wasn’t that big of a deal, in fact it was pretty important – he said I or a tax professional would need to run the numbers and compare them with my broker’s to make sure everything added up.

Moreover, Ringbauer told me that a few numbers on my 1099 showed that I had made some taxable moves that I might not have made. ChatGPT had said nothing about them; probably because I didn’t think to ask.

“The question is: Does the taxpayer have the necessary understanding of the documents they are looking at to understand and correct any items that need to be addressed?” says Ringbauer. “From what I understand, most of our customers do not do this.”

Overall, the error I encountered was minor. ChatGPT told me something was “almost certainly” true when I really had some homework to do. But more importantly, the advice I received seemed so solid and was given so quickly that I was willing to apply and risk making a mistake.

For people who don’t specialize in tax law, this can be a big problem, says JT Eagan, a clinical assistant professor of accounting at Purdue University.

“AI will convince you that the sky is green. It’s very convincing,” he says, referring to a time when a chatbot incorrectly answered one of the tax questions it gave Eagan’s students. “He gave me the answer that the mechanics were perfect, but I had to step back and say, ‘Well, you’re wrong.'”

It is not the case that artificial intelligence chatbots want to deceive you. In fact, in recent years, many of the leading models have gotten better at letting users know they don’t have the exact answer to certain prompts or are wandering around gray areas, says Jordan Wilson, founder of AI strategy firm Everyday AI.

In fact, at the beginning of our conversation, ChatGPT told me: “If anything gets too specific or risky, I’ll mark places where you can ask for the CPA’s input.”

“But by default large language models are trained as auxiliaries,” says Wilson. This means that they often seem very confident and “most of the time you will encounter hallucinations.”

And they can be delivered with surprising confidence. “This is a very simple return with one stock plan wrinkle,” ChatGPT told me. “You DO NOT need a CPA.”

With that in mind, experts say you should be very careful before using AI to help with your taxes.

“If you make a mistake when using AI to pay your taxes, you could get in trouble with the IRS,” says Wilson. “And saying ‘AI made me do this’ is not a valid excuse.”

If you plan to use AI as a tool during tax season, here’s what professionals say you should keep in mind.

Check which model you are using

No two LLMs are exactly the same, Wilson says, and even if you have a favorite AI company, the firm will often offer different models with different capabilities.

For complex topics like taxes, he says, you’ll want to be in a “thinking” pattern that breaks down difficult topics and demonstrates their work by coming up with step-by-step solutions.

Even if you’re not using the free version of a particular chatbot, “most of these companies, by default, will redirect you to their faster models that aren’t as smart,” says Wilson.

AI companies like Anthropic and OpenAI have tiered pricing that gives you access to differently powerful versions of their AI chatbots. Even if you are at a higher level, the model you are using may not be calibrated for complex topics at the beginning. When I first opened my company’s paid version of ChatGPT, it defaulted to “Instant” mode for casual chats. To get into “thinking” mode, I had to click on a menu at the top of the chat.

Another factor worth checking before uploading your documents is whether the AI ​​firm is using your information to train its model. Wilson says you can usually turn off model training when using paid versions, but users of free versions should think twice before sharing personal information.

Your information won’t be made public, he says, “but companies can use all kinds of data if you have a free plan to train their models. And that’s probably not something you want, especially when it comes to sensitive information like your bank account information, your Social Security number.”

Understand AI’s and your limitations

Experts say that artificial intelligence can reach the wrong answer even if it follows the correct process. One reason for this may be that the model is based on old data, Wilson says. The data used to train a model can be months old in some cases, leaving it without the most up-to-date tax information, he says. This can force a model to get updated information from the internet.

“Where will he verify this information? Will he find an article from 2024, 2025 or 2026?” Wilson says. “The difference can obviously wildly affect the results and accuracy of whatever you’re using it for.”

In some cases, AI may provide a perfect answer to a tax question, but that doesn’t mean it will be the right answer for you. This is because certain tax rules may or may not apply to you, given the particular nuances of your situation.

To give a simplified example, consider a taxpayer wondering whether he can deduct his dog. Ask a CPA and they’ll tell you “it depends,” Eagan says. This is because while you can’t write off a pet, under certain circumstances you can deduct the cost of a sighted dog as a medical expense.

Ask the question to an AI chatbot and “it [may] Start with ‘Yes,'” says Eagan. “The challenge you face is how many people stop reading after they get the answer they want to hear.”

Looking across the tax code, it’s not hard to see how users might have a hard time getting an accurate return by relying on AI for answers. Even if an AI answers a question correctly or makes an accurate calculation, it’s limited to what it knows about you, Wilson says. Without a holistic approach, you doom these models to failure, he says.

None of this means that models will automatically fail when faced with tax issues. You have to ask yourself if you are equipped to give the model all the relevant information it needs and ask for all the relevant follow-ups. If the answer is no and there’s too much money at stake, it’s wise to consult a professional, Wilson and other experts say.

“I think one of the biggest missteps that most people make, even those who feel pretty comfortable using AI, is thinking: ‘Okay, as long as I input some of my documents and give it an appropriate context, it’ll do a good job at the output,’” says Wilson. “But one actually has to do a lot of hands-on work to make sure you’re giving the model the right context.”

Do you want to lead with confidence and bring out the best in your team? Take CNBC’s new online course, How to Become a Stand Out Leader?. Expert trainers share practical strategies to help you build confidence, communicate clearly, and motivate others to do their best.