Yes, he does. But I don’t think that invalidates everything he writes.
If you think about it, after a certain complexity level, even in animals, an increasing number of neurons gives rise to behaviour (not only improving existing behaviours). A dog or cat will recognise its name, when called, can express feelings and so on, while a ringworm will not / cannot. In a certain sense, the LLMs ‘understand’ stuff. They may be very limited (I was shocked to see how badly they play simple games like tic-tac-toe, for example).
Here is what I mean by ‘some limited understanding of the world’:
Prompt:
Three objects, 'A', 'B' and 'C'
are placed on a 100 cm wide table
along a straight line spanning the width of the table.
None of the objects falls off the table.
Object 'A' is directly on the left side.
Object 'B' is 80 cm from 'A'.
Object 'C' is 30 cm from 'B'.
Work in steps.
What is the distance between 'A' and 'C'?
Output:
Let me solve this step by step.
1. First, let's understand what we know:
* The table is 100 cm wide
* Object 'A' is on the left side
* Object 'B' is 80 cm from 'A'
* Object 'C' is 30 cm from 'B'
2. Since 'A' is on the left side and 'B' is 80 cm from 'A':
* 'B' must be 80 cm to the right of 'A'
3. Now, 'C' is 30 cm from 'B', and it could be either:
* 30 cm to the right of 'B', or
* 30 cm to the left of 'B'
4. If 'C' is 30 cm to the right of 'B':
* Distance from 'A' to 'C' = 80 + 30 = 110 cm
* But this is impossible because the table is only 100 cm wide
5. Therefore, 'C' must be 30 cm to the left of 'B'
* Distance from 'A' to 'C' = 80 - 30 = 50 cm
The distance between 'A' and 'C' is 50 cm.
And yes, it is stupid, because if I omit the ‘None of the objects falls off the table’ part, then the 3rd part becomes:
3. 'C' is 30 cm from 'B':
Since we're working along a straight line
And moving from left to right
'C' must be 30 cm to the right of 'B'
To find the distance between 'A' and 'C':
Distance from A to C = Distance from A to B + Distance from B to C
Distance from A to C = 80 cm + 30 cm
Distance from A to C = 110 cm
Therefore, the distance between 'A' and 'C' is 110 cm.
All very logical, except for:
- ignoring that all objects are on the table
- inventing that we are working from left to right, which I never included in my prompt.
Keeping the first line of the prompt as in the 2nd example, but changing the question also results in the right answer:
Three objects, 'A', 'B' and 'C'
are placed on a 100 cm wide table
along a straight line spanning the width of the table.
...
Going left to right,
what is the order of the objects as laid out on the table?
Answer:
...
Object 'C' is 30 cm from 'B':
C's position = 80 + 30 = 110 cm
However, since the table is only 100 cm wide, 'C' cannot be to the right of 'B'
'C' must be between 'A' and 'B'
Therefore, going left to right, the order must be:
A, C, B
It does seem (luckily) that improvements come at exponential costs (I found a chart somewhere, but don’t know where). I do hope to keep my job.
