Why Is Escaping Data Considered So Magical?

[Robert Kern]

I thought it was well-known that the solution is *not* to try to
sanitize the input -- it's to switch to an interface which doesn't
involve generating an intermediate executable. In the Python example,
that would be something like os.popen2(['zcat', '-f', '--', untrusted]).

That’s what I mean. Why do people consider input sanitization so hard?

It's not hard per se; it's just repetitive, prone to the occasional mistake,
and, frankly, really boring. When faced with things like that, we do what we do
everywhere else in programming: wrap up the repetitive bits into a simpler
library API and use that everywhere. Wrapping up the escaping code into
SQLString is a step in that direction. However, the standard SQL
parameterization in most of the DB protocols or SQLAlchemy's query construction
removes even more repetition and unnecessary typing. There's just no point in
not using it.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Nobody]

That’s a content-free claim. Why? Because it applies equally to everything.
Replace “quoting†with something like “arithmeticâ€, and you’ll
see what I mean:

If you omit the arithmetic, the program is likely to fail in very
obvious ways. Escaping is "almost" an identity function, which makes it
far more likely that omission or repetition will go unnoticed.

Nonsense. It only needs to be done at the boundary to the appropriate
component (MySQL, HTML, JavaScript, whatever).

That assumes that you have a well-defined "boundary", which isn't
necessarily the case.

In any case, you're still trying to make arguments about whether it's easy
or hard to get it right, which completely misses the point. Eliminating
the escaping entirely makes it impossible to get it wrong.

 

[Nobody]

To bring this back to something remotely Python related, the point of
all this is that security is hard.

Oh, this isn't solely a security issue.

Ask anyone with a surname like O'Neil, O'Connor, O'Leary, etc; they've
probably broken a lot of web apps *without even trying*.

 

[Roy Smith]

It's not hard. It's just begging for a visit from the fuckup fairy.

QOTD?

 

[D'Arcy J.M. Cain]

Ask anyone with a surname like O'Neil, O'Connor, O'Leary, etc; they've
probably broken a lot of web apps *without even trying*.

At least it isn't a problem with the first name field. Oh, wait...

 

[Lawrence D'Oliveiro]

On 06/25/2010 07:49 PM, Lawrence D'Oliveiro wrote:
...

I see that you published my unobfuscated e-mail address on USENET for all to
see. I obfuscated it for a reason, to keep the spammers away. I'm assuming
this was a momentary lapse of judgement, for which I expect an apology.
Otherwise, it becomes grounds for an abuse complaint to your ISP.

 

[Lawrence D'Oliveiro]

Your example from the first post of the thread rewritten using sqlalchemy:

conn.execute(
items.update()
.where(items.c.inventory_nr == modify_id)
.values(
dict(
(field[0], Params.getvalue("%s[%s]" % (field[1],
urllib.quote(modify_id))))
for field in [
(items.c.class_name, "modify_class"),
(items.c.make, "modify_make"),
(items.c.model, "modify_model"),
(items.c.details, "modify_details"),
(items.c.serial_nr, "modify_serial"),
(items.c.inventory_nr, "modify_invent"),
(items.c.when_purchased, "modify_when_purchased"),
... you get the idea ...
(items.c.location_name, "modify_location"),
(items.c.comment, "modify_comment"),
]
)
)
.values(last_modified = time.time())
)

Doesn't seem any less flexible to me, plus you don't have to worry
about calling your SQLString function at all.

Except I only needed two calls to SQLString, while you need two dozen
instances of that repetitive items.c boilerplate.

As a human, being repetitive is not my job. That’s what the computer is for.

 

[Lawrence D'Oliveiro]

It's not hard. It's just begging for a visit from the fuckup fairy.

That’s the same fallacious argument I pointed out earlier.

 

[Lawrence D'Oliveiro]

If you omit the arithmetic, the program is likely to fail in very
obvious ways. Escaping is "almost" an identity function, which makes it
far more likely that omission or repetition will go unnoticed.

Maybe you need to go back and reread my original posting. The SQLString
routine doesn’t just escape special characters, it generates a full MySQL
string literal, complete with quotation marks. That makes it rather more
likely for a syntax error to occur if I forget to use it, don’t you think?

That assumes that you have a well-defined "boundary", which isn't
necessarily the case.

It’s ALWAYS the case.

In any case, you're still trying to make arguments about whether it's easy
or hard to get it right, which completely misses the point. Eliminating
the escaping entirely makes it impossible to get it wrong.

Except nobody has yet shown an alternative which is easier to get right.

 

[Aahz]

I see that you published my unobfuscated e-mail address on USENET for all to
see. I obfuscated it for a reason, to keep the spammers away. I'm assuming
this was a momentary lapse of judgement, for which I expect an apology.
Otherwise, it becomes grounds for an abuse complaint to your ISP.

You are double daft. First, I completely disagree with you about it
being abuse; from my POV anyone posting to Usenet should do so with an
unobfuscated address. Secondly, you are wrong about Tim publishing your
address unless you intended to follow up to a completely different post,
and you owe *him* an apology for a false accusation.

 

[Chris Rebert]

I see that you published my unobfuscated e-mail address on USENET for all to
see. I obfuscated it for a reason, to keep the spammers away. I'm assuming
this was a momentary lapse of judgement, for which I expect an apology.
Otherwise, it becomes grounds for an abuse complaint to your ISP.

Will you give it a rest already with these threatening messages? Why
are you still using this only-partially-obfuscated address with USENET
anyway? This has happened twice before, it will doubtless happen yet
again. Just use an /entirely invalid/ From address like some other
posters do.

I can't believe you have a form letter for this...

Regards,
Chris

 

[Tim Chase]

I see that you published my unobfuscated e-mail address on USENET for all to
see. I obfuscated it for a reason, to keep the spammers away. I'm assuming
this was a momentary lapse of judgement, for which I expect an apology.
Otherwise, it becomes grounds for an abuse complaint to your ISP.

I'm sorry...you've got your knickers in a knot? That your spam
filters seem to be insufficient? That you don't have a custom
throwaway address for such public dialogs? For preventing an
"undeliverable" bounce message that your bogus address would have
caused (if your mail provider is RFC-compliant; though your mail
provider may kindly be breaking RFC by disabling "undeliverable"
responses to prevent back-scatter spam)?

Is the abuse charge "waah, he replied to my actual email rather
than the false one I spoofed"?

I'm not sure an abuse complaint to my ISP would net you anything
since the exact out-bound headers show nothing abusive, only the
correcting of an invalid TLD to prevent a bounce (and a distinct
lack of USENET references in the original message that went to
you and CC'ed (e-mail address removed)).

Having regularly used (e-mail address removed) unobfuscated
for easily over 5 years, the spam to this address has been almost
negligible (or so effectively dealt with by Thunderbird's spam
filters that I've never noticed it).

-tkc

 

[Stephen Hansen]

I see that you published my unobfuscated e-mail address on USENET for all to
see. I obfuscated it for a reason, to keep the spammers away. I'm assuming
this was a momentary lapse of judgement, for which I expect an apology.
Otherwise, it becomes grounds for an abuse complaint to your ISP.

Wow.

Way to be a douchebag.

I was going to say something about the realities of this forum and its
dual-nature and conflicting netiquette and on. But I decided it really
just had no point.

So, I'm left with: wow. You kinda suck*, man.

--

... Stephen Hansen
... Also: Ixokai
... Mail: me+list/python (AT) ixokai (DOT) io
... Blog: http://meh.ixokai.io/

P.S. *Then again, I'm fairly sure anytime someone has a form letter
which contains the words, "I expect an apology", there's some personal
suck going on.

 

[geremy condra]

Er? On what grounds would you complain to their ISP? You might consider
the person rude, but that's not grounds for an abuse complaint. What
part of their ISP's terms of service do you think they have abused by
de-obfuscating information you freely posted to the internet?

I routinely post my email on this and other mailing lists and have yet
to get a piece of spam in my inbox as a result. I suggest you get a
better spam filter rather than expecting the rest of the universe to
annoy itself for your benefit.

Geremy Condra

 

[Lawrence D'Oliveiro]

Ask anyone with a surname like O'Neil, O'Connor, O'Leary, etc; they've
probably broken a lot of web apps *without even trying*.

Last I checked, I couldn’t post comments on freedom-to-tinker.com.

 

[Lawrence D'Oliveiro]

I recently fixed a bug in some production code. The programmer was
careful to use snprintf() to avoid buffer overflows. The only problem
is, he wrote something along the lines of:

snprintf(buf, strlen(foo), foo);

A long while ago I came up with this macro:

#define Descr(v) &v, sizeof v

making the correct version of the above become

snprintf(Descr(buf), foo);

 

[Ian Kelly]

Except I only needed two calls to SQLString, while you need two dozen
instances of that repetitive items.c boilerplate.

As a human, being repetitive is not my job. That’s what the computer is for.

Then why do you have every parameter prefixed with "modify_"?

But seriously, if that bothers you, then fold the "items.c." portion
into the generator expression with a getattr call. Or just change
them back to the same strings you had originally, and sqlalchemy will
be just as happy to accept them as-is.

Cheers,
Ian

 

[Kushal Kumaran]

A long while ago I came up with this macro:

   #define Descr(v) &v, sizeof v

making the correct version of the above become

   snprintf(Descr(buf), foo);

Not quite right. If buf is a char array, as suggested by the use of
sizeof, then you're not passing a char* to snprintf. You need to lose
the & in your macro.

 

[Lawrence D'Oliveiro]

Not quite right. If buf is a char array, as suggested by the use of
sizeof, then you're not passing a char* to snprintf.

What am I passing, then?

 

[Lawrence D'Oliveiro]

Then why do you have every parameter prefixed with "modify_"?

Touché . Actually it’s because the same form can be used to add a new
record to the table, so there’s a separate set of input fields for that.

But seriously, if that bothers you, then fold the "items.c." portion
into the generator expression with a getattr call. Or just change
them back to the same strings you had originally, and sqlalchemy will
be just as happy to accept them as-is.

All this trouble, and it only gets rid of 2 of the 3 instances of data-
escaping in the example.