FRESH

Hacker News

A Tale of Two AI Failures: Debugging a Simple Bug with LLMs

7 points by slederer

by lillesvin

2 subcomments

I know it's kinda besides the point and I don't know what language this was being done in, but I don't personally know any language where
```
    String signature = "POST" + "\n" + "/api/v1/..."
```
and
```
    String signature = "POST\n/api/v1/..."
```
don't result in identical variables, so I'm a bit puzzled why that would result in an error.
However, there's a quoting error in the failing example where the double quotes in the JSON body aren't properly escaped:
```
    String signature = "POST" + "\n" + "/api/v1/query" + "\n" + token + "\n" + timestamp + "\n" + "{"body":"content"}"
```
It may just be the example that's not correctly formatted, but the other (working) example does in fact escape the double quotes in the JSON. I guess, depending on how forgiving the used language is with quoting, that could also be the source of the error?

by nebster

0 subcomment

I... still don't understand the issue. It looks like both examples in the table would evaluate to the same thing. Am I missing a stray "\n"?

by DylanSp

0 subcomment

Echoing the others who say they can't understand the bug/difference; only thing I can think of is that the input string needed the escape sequence for a newline in it? So the correct code would be written as
```
    "POST" + "\\n" + ...
```

by kichik

1 subcomments

Not exactly the point of this article, but it would be cool if APIs like this can return the expected signed string for debugging. It would have to be properly limited for security. But if the API is expecting non-standard signatures, it could help developers with better debugging tools.

by fernly

0 subcomment

agree, I feel dumb but don't see subtle issue.
Also when copy/pasting into Python to try it, I got an error because \“ is in fact U+201C not an ASCII quote. (Surely that's not the subtle issue?)

by thehappypm

0 subcomment

tl;dr: custom, naïve Concatenation formatting implementation can cause bugs