Do LLMs reason?

LLMs sure seem to reason. They'll show, step-by-step, how they reach a conclusion.

However, there are two things to keep in mind:

1. When it shows you the reasoning in a step-by-step way, that is there for your benefit. It is not a description of a linear train of thought by the LLM. LLMs typically do their analysis of input basically in one pass. Its analysis is largely pattern matching, not linear thinking.. And output is similarly generated in a single pass. The step-by-step description is mostly generated after the analysis as a plausible explanation, not a description of the LLM's "thought process".
2. LLMs don't have self-introspection. They can directly contradict their own analysis steps without realizing it.

Here's an example. My prompt:

I'm looking at a wireshark dissection of a customer pcap, and the ethernet "protocol type" field is 0x8100, which it identifies as 802.1Q VLAN. It's followed by the VLAN tag and the "real" protocol type, which is 0x0800, or IP. So far, so good.

But what's the "trailer", which appears to be glued onto the end of the packet? I don't see that in pcaps in our local lab, which does not use VLANs.

Claude's response:

The "trailer" you're seeing is likely padding added to meet Ethernet's minimum frame size requirement of 64 bytes (including the 4-byte FCS/CRC).

When you add a VLAN tag, you're inserting 4 extra bytes into the frame structure. If the original payload was already close to the minimum size, this can push some frames below the 64-byte minimum. The Ethernet hardware (typically the switch or NIC) automatically pads these frames with zeros to meet the minimum size requirement.

It continues from there. But anybody can see that adding a VLAN tag will NOT push the frame size below the minimum! It made a reasoning error!

But I'm not so sure it did. I think it works in two phases: analysis and output generation. During the analysis phase, it is pattern matching input data against its training data. It got strong pattern matches related to packet size, vlan tags, and padding. These concepts cluster when talking about packets, so the analysis reasonably concluded that these are related to the issue I described.

Then it generated output. But it's not analyzing its own output, it's generating. It put into words the conceptual connections it made, and you see what came out.

As an experiment, I followed up with the prompt:

I want you to analyze your conclusions critically and tell me if you see any logical contradictions in what you said.

Here's Claude's response:

You're right to push back. I made a logical error.

I said the VLAN tag adds 4 bytes and might push frames below the 64-byte minimum, requiring padding. But that's backwards - adding bytes would push frames above the minimum, not below it. Frames that were already at or above 64 bytes wouldn't suddenly need padding just because you added 4 more bytes.

So during analysis, it can apply reasoning. But it can only reason on the input data, not it's own conclusions.

Interestingly, Claude has an "extended thinking" mode. In this mode, it does seem to be performing a step-by-step analysis, which I suspect is just a loop where it feeds its intermediate output back into its analysis engine. I've seen it start down one path, see a contradiction, and backup and try another path. A form of "self reflection" implemented as a loop.

I tried extended thinking with my original prompt, and while its "internal monologue" mentioned the vlan tag, it went straight to FCS, which is the correct answer as it happens.

Aside: I'm sometimes tempted to use "extended thinking" more than I do. But I suspect it uses a LOT more energy, and most of the time I don't think it would produce better results for the kinds of queries I make. So I mostly rely on my own reasoning to catch Claude's mistakes. Also note that I strongly suspect that "extended thinking" won't solve the hallucination problem. Most hallucinations I've seen are not holes in logic, it's making connections that make sense in the abstract, but aren't actually factual. Extended thinking probably won't help that.

In conclusion, Claude does reason, but does so at specific phases of its processing which can still make it vulnerable to logical mistakes.

P.S. - for those network engineers out there, the real issue is that my customer pcap was from a Corvil device, which captures the FCS. Normal wireshark does not get FCS from the NIC/driver, so Wireshark isn't used to seeing it. Hardware packet capture typically does include it. So my suspicion that it was related to VLAN was incorrect; it had nothing to do with that.