Bug Tracker

Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#13491 closed bug (notabug)

Cannot suppress X-Requested-With on same origin requests

Reported by: nickshanks@… Owned by:
Priority: undecided Milestone: None
Component: unfiled Version: 1.8.3
Keywords: Cc:
Blocked by: Blocking:

Description (last modified by jaubourg)

I am trying to globally remove the X-Requested-With header from all AJAX requests to my own site (since I don't care whether it's AJAX or not, and I see no need to send the additional request header bytes).

I am not the first to want this. To wit: This Fiddle does not work - http://jsfiddle.net/AUGQJ/1/ It was referenced from http://stackoverflow.com/questions/6432320/how-to-make-an-ajax-request-look-like-a-regular-normal-request

It instead sends a header with an empty value.

The code at fault is:

// X-Requested-With header
// For cross-domain requests, seeing as conditions for a preflight are
// akin to a jigsaw puzzle, we simply never set it to be sure.
// (it can always be set on a per-request basis or even using ajaxSetup)
// For same-domain requests, won't change header if already provided.
if(!s.crossDomain && !headers["X-Requested-With"])
  headers["X-Requested-With"] = "XMLHttpRequest";

which is immediately followed by

for(i in headers)
  xhr.setRequestHeader(i, headers[i]);
xhr.send();

This means there is NO WAY to suppress the header from being sent on same-origin requests.

I would like to request one or two of these three changes which would allow this:

  1. some flag that can be set to true in .ajaxSetup(), default to send (i.e. no change to current behaviour by default):
if(!s.crossDomain && !headers["X-Requested-With"] && !s.suppressRequestedWithHeader)
  headers["X-Requested-With"] = "XMLHttpRequest";
  1. some flag that can be set to true in .ajaxSetup(), default to not send (i.e. reduces unnecessary internet traffic):
if(!s.crossDomain && !headers["X-Requested-With"] && s.sendRequestedWithHeader)
  headers["X-Requested-With"] = "XMLHttpRequest";
  1. [not exclusive with 1 or 2] Validate that headers set have non-empty values:
for(i in headers)
  if(headers[i] != '')
    xhr.setRequestHeader(i, headers[i]);
xhr.send();

This will allow jqXHR.setRequestHeader('X-Requested-With','') to suppress the header (rather than use a boolean).

I would like #2 and #3 but I'd understand if you went with #1 and #3. Only doing #1 would be the most conservative and least helpful (barring of course, not fixing this bug!).

Change History (7)

comment:1 Changed 7 years ago by anonymous

Additionally, your bug submission system failed to accurately reproduce what I typed. It should have saved the field value verbatim to the database, then on production of HTML output, escaped ampersands, less-than signs, and wrapped it in a PRE element. Instead it tried to do all kinds of fancy reformatting, resulting in a colossal mess. Congratulations.

comment:2 in reply to:  1 ; Changed 7 years ago by jaubourg

Description: modified (diff)
Resolution: notabug
Status: newclosed

Replying to anonymous:

Additionally, your bug submission system failed to accurately reproduce what I typed. It should have saved the field value verbatim to the database, then on production of HTML output, escaped ampersands, less-than signs, and wrapped it in a PRE element. Instead it tried to do all kinds of fancy reformatting, resulting in a colossal mess. Congratulations.

I guess you missed the WikiFormatting link. Congratulations.

I fixed your post for you.

The stackoverflow question you reference has nothing to do with X-Requested-With. It's just someone foolishly trying to bypass the same origin barrier by overriding the Origin header (which is impossible and pointless as is clearly stated in the selected answer).

jQuery always sent the X-Requested-With header for same-origin request (oldest version I could check this in is 1.1.2). It is completely harmless, as it will be ignored server-side unless some code specifically checks for it, and only costs 31 bytes. It won't go for backward compatibility purpose (because server-side code may rely on it).

comment:3 in reply to:  2 ; Changed 7 years ago by nickshanks

Replying to jaubourg:

I guess you missed the WikiFormatting link. Congratulations.

I fixed your post for you.

Thanks. It's appreciated. Yeah, I did miss that link—it's so incy-weency and off on the right where my eyes never wander. That, and the lack of a real-time javascript-powered preview, which would have made things obvious before I hit submit.

The stackoverflow question you reference has nothing to do with X-Requested-With. It's just someone foolishly trying to bypass the same origin barrier by overriding the Origin header (which is impossible and pointless as is clearly stated in the selected answer).

I know. I was just referencing it as that's how I came around to trying this out.

jQuery always sent the X-Requested-With header for same-origin request (oldest version I could check this in is 1.1.2). It is completely harmless, as it will be ignored server-side unless some code specifically checks for it, and only costs 31 bytes. It won't go for backward compatibility purpose (because server-side code may rely on it).

I am not asking you to affect backwards compatibility (that was option 2, which you may still to consider for jQuery 2.x) but was only requesting the addition of an opt-in boolean so that we (we, who don't look for the header) can eliminate those 31 bytes of useless data between our server and our client. Smaller headers means fewer packets on average, which means fewer chances for users to get left hanging when packets go AWOL. It's not really about saving 31 bytes on megabit-per-second connections, but about serving up responses with fewer failures.

The main reason I filed this as a jQuery bug is so that I don't have to use a custom jQuery script, and can instead use a common CDN-hosted 'standard' version with all the multi-site cache priming goodness that comes with.

This request has merit and should be dismissed out-of-hand just by closing the bug and hoping myself and others writing same-origin web apps will just go away. I would like to hear the opinions of a few of jQuery's other developers before conceding and hosting my own altered copy.

comment:4 in reply to:  3 ; Changed 7 years ago by jaubourg

Replying to nickshanks:

Replying to jaubourg:

I guess you missed the WikiFormatting link. Congratulations.

I fixed your post for you.

Thanks. It's appreciated. Yeah, I did miss that link—it's so incy-weency and off on the right where my eyes never wander. That, and the lack of a real-time javascript-powered preview, which would have made things obvious before I hit submit.

If you want to donate of your time and help on the bug tracker, the help will be more that appreciated. I too find this site hard to navigate and I do it daily. However, I do realize it's been done by volunteers that donated of their time so that people like me can fix bug for people like you. If you can do a better job, please, do it.

There's a preview button though and it's pretty easy to find in the interface.

The stackoverflow question you reference has nothing to do with X-Requested-With. It's just someone foolishly trying to bypass the same origin barrier by overriding the Origin header (which is impossible and pointless as is clearly stated in the selected answer).

I know. I was just referencing it as that's how I came around to trying this out.

So you didn't have any actual need first hand? We're just nitpicking here? No "it's breaking my app" bugs nor "look how enhanced the ajax module is now" feature requests around?

jQuery always sent the X-Requested-With header for same-origin request (oldest version I could check this in is 1.1.2). It is completely harmless, as it will be ignored server-side unless some code specifically checks for it, and only costs 31 bytes. It won't go for backward compatibility purpose (because server-side code may rely on it).

I am not asking you to affect backwards compatibility (that was option 2, which you may still to consider for jQuery 2.x) but was only requesting the addition of an opt-in boolean so that we (we, who don't look for the header) can eliminate those 31 bytes of useless data between our server and our client.

Actually, you are asking for backward compatibility breakage. Any backend dealing with jQuery knows this header will be defined, if we provide an opt-in boolean option, a third-party script will be able to change it globally using ajaxSetup thus breaking the assumption of the backend. It would spread unless specifically specified in the options of each request.

That's one of the reasons why we don't add options lightly in ajax.

Smaller headers means fewer packets on average, which means fewer chances for users to get left hanging when packets go AWOL. It's not really about saving 31 bytes on megabit-per-second connections, but about serving up responses with fewer failures.

Do you have any actual data that shows how many packets we're talking about on average? Or is this just some mental exercise in micro-optimization?

I'm sorry, but you still fail to show how mandatory the feature is to your applications and, more importantly, to others.

You don't check for the header, fine, so its presence doesn't do you any harm anyway.

The main reason I filed this as a jQuery bug is so that I don't have to use a custom jQuery script, and can instead use a common CDN-hosted 'standard' version with all the multi-site cache priming goodness that comes with.

I'm sorry, but if you use a custom version of jQuery to remove an innocuous header that's been here for at least 8 major versions without anyone complaining, you're doing it wrong (tm).

I actually seriously doubt you did so or actually intend to do so. Especially if you didn't try it out before you saw the comment on stackoverflow.

Also, if you do some research on the bug tracker, you'll realize that we had numerous bugs for when the header wasn't set (regressions), yet none regarding removing it, except indirectly in the context of CORS requests to avoid preflight (now, avoiding preflight does smooth user experience).

So far, you're the only person who has asked for the possibility to remove the header for all types of request on this bug tracker... for 8 major versions.

This request has merit and should be dismissed out-of-hand just by closing the bug and hoping myself and others writing same-origin web apps will just go away.

I'm sorry, but the request doesn't have much merit. You admitted you saw a comment in an unrelated stackoverflow question, tried it out and are now compaigning for a new ajax option. All the while, you fail to provide any tangible need nor any kind of actual data to support this would enhance user experience.

You talk of others but, again, you're the only one who ever expressed this need.

I would like to hear the opinions of a few of jQuery's other developers before conceding and hosting my own altered copy.

I'm sure other jQuery's devs will appreciate your tone as much as I do.

comment:5 in reply to:  4 ; Changed 7 years ago by nickshanks

Replying to jaubourg:

If you want to donate of your time and help on the bug tracker, the help will be more that appreciated...

Understood. I'll look upstream to see if there's already a feature request for that at trac.

There's a preview button though and it's pretty easy to find in the interface.

Yes, I ignored that and shouldn't have done.
Like spell-checking, unless it happens inline as I type, I am prone to just skipping optional steps. I presume such is typical of human behaviour, since spelling errors before the advent of check-as-you-type were so common that realtime spell-checking had to be invented. I do the same thing on Wikipedia edits, never using the Preview button and then correcting it if it turns out unexpectedly.

So you didn't have any actual need first hand? We're just nitpicking here? No "it's breaking my app" bugs

Nothing is specifically broken that removing this would guarantee to fix (although see below for what is broken). I had assumed it would be a simple thing to add a boolean and an if() block around these two lines of code. My despondence grew from that naïve assumption.

nor "look how enhanced the ajax module is now" feature requests around?

not sure what you meant by this.

Actually, you are asking for backward compatibility breakage. Any backend dealing with jQuery knows this header will be defined, if we provide an opt-in boolean option, a third-party script will be able to change it globally using ajaxSetup thus breaking the assumption of the backend. It would spread unless specifically specified in the options of each request.

That's one of the reasons why we don't add options lightly in ajax.

That's a scenario that I hadn't considered as it is not part of my workflow. It's a good reason for closing this ticket, sufficient to make me withdraw my request for a second developer's opinion.

Do you have any actual data that shows how many packets we're talking about on average?

A simple approximation can be computed. Assume a packet size of 1500 bytes and that response sizes follow a flat distribution,† the chance of an extra packet being needed because of this header are 31/1500 or about 2%. For all the people in my company who work with the web app I am responsible for, and extrapolating from the complaints I hear from a vocal few who otherwise seem representative, we see perhaps 100 timeouts a day on response retrieval (most of the app's users are on 3G company-issued iPhones, driving around the country with potentially unreliable data connections). I have no idea how many of those are due to packet loss though.‡

Eliminating unnecessary headers, including this one, was just part of a wider push I have been doing to improve caching and reduce response size of the web app. I have managed to more than halve the size of many responses during this process. All those 2%s add up!

† I have not produced a bell-curve distribution graph for my own app, but the global average for websites is about 6k for text/html responses. I also realise that there would be a non-zero minimum as even a 204 response carries certain HTTP headers too.

‡ Our web host is in the US, which doesn't help, as every request has to cross the Atlantic, my boss won't move to a different hosting company, and our BT-provided office DSL line has an uptime considerably less than a web hosting company can offer, so we don't want to host it in-house. Timeouts on GET requests are the least severe, and non-idempotent ones like POST and PATCH are the worst, since the user doesn't know the state of the resource with certainty. Often when I investigate these non-GET timeouts, I find that the resource does get updated but the client never receives the HTML page saying 'done'. I suspect the GET timeouts are similar but haven't investigated those since they are of less consequence.

Or is this just some mental exercise in micro-optimization?

As I say, this was not an individual, specific micro-optimisation but just a small part of an onslaught to reduce network bandwidth and improve successful response rates. I was only intending to spend 1-3 minutes on Google to discover where the switch was to toggle this header off (under the assumption that there was such a switch). That amount of time was worth spending for the expected benefits.

I'm sorry, but you still fail to show how mandatory the feature is to your applications and, more importantly, to others.

It is not. I wasn't trying to show that it was mandatory (and thought I had conveyed such).

You don't check for the header, fine, so its presence doesn't do you any harm anyway.

I wouldn't jump to that conclusion just yet. I will let the other changes made recently bed in for a while, and see if our response timeout rate drops. If it does, my suspicions that packet loss is the primary cause will be strengthened.

The main reason I filed this as a jQuery bug is so that I don't have to use a custom jQuery script, and can instead use a common CDN-hosted 'standard' version with all the multi-site cache priming goodness that comes with.

I'm sorry, but if you use a custom version of jQuery to remove an innocuous header that's been here for at least 8 major versions without anyone complaining, you're doing it wrong (tm).

What other options do I have? I am willing to explore them. Anything to help get rid of this header.

I actually seriously doubt you did so or actually intend to do so.

Doubt I did what? We were hosting our own (unmodified) copy of jQuery up until August last year when I switched to Google's CDN. The trade-off was marginal. An extra DNS lookup, establishing a network connection for just a single request (no benefit from keep-alive), and introducing another server that could potentially go down and bring the company to a halt, vs. closer geo-location (remember, the rest of the web app is served from 5000 miles away) and intangible potential cross-site cache priming. I even imported HTTP Archive's 21 GB (gzipped!) SQL database to find out which was the most common version to link to (1.7.2 at the time, inspired by Steve Webster) to improve chances of a cache hit. Google only uses long Expire/max-age headers on the three-digit versions; the 'latest 1.x' versioning scheme uses just 1hr expire times, and would have resulted in hundreds of 304s per day, and since this is often the only request to Google's CDN, eliminating the entire network connection by specifying a specific version can only help.

Especially if you didn't try it out before you saw the comment on stackoverflow.

I found that page after Googling for ways to remove this header. The failure of the technique used in the Stack Overflow answer is what led to me looking at the jQuery code, and consequently, filing this feature request.

I'm sorry, but the request doesn't have much merit. You admitted you saw a comment in an unrelated stackoverflow question,

The question on that page was unrelated, yes, but one of the answers was not, and it was a comment on that answer that linked to the jsfiddle, which I only provided as your ticket report form asked for one.

tried it out and are now compaigning for a new ajax option. All the while, you fail to provide any tangible need nor any kind of actual data to support this would enhance user experience.

Hopefully I have explained why I felt the need to ask for this. For 2% it was worth the 5 minutes required to write the initial ticket. It has now spiralled into a much bigger use of my time and yours (for which I am grateful, even if that's difficult to express in a debate like this).

You talk of others but, again, you're the only one who ever expressed this need.

Fair enough. I am the first. Yay me.

I would like to hear the opinions of a few of jQuery's other developers before conceding and hosting my own altered copy.

I'm sure other jQuery's devs will appreciate your tone as much as I do.

Tone? Apart from 1. being a bit pissed off at all my formatting got eaten (and not realising it would be so easily fixable) and thus my knee-jerk first comment, and 2. a feeling that the swiftness with which the bug was closed, having had only one person look at it, meant that it was not being taken seriously. I felt it had been dismissed out-of-hand without good reason (you didn't say why it would break compatibility in your first comment).

Please don't try to read too much into my choice of words earlier. I report bugs in software only when I have reached the end of a frustrating road trying to get the software to do as I wish it to (jQuery being the latest in a *very* long list). Thus I am already annoyed/angry/upset/pissed off by the time I click the "New Ticket" button and vent into a text field. My all-time worst job would be to work on the customer support phones for an ISP or utility company and have to deal with angry customers day in, day out!

comment:6 in reply to:  5 Changed 7 years ago by jaubourg

Understood. I'll look upstream to see if there's already a feature request for that at trac.

The best place to report issues regarding the bug tracker itself is here. It's a private repo but we could probably work something out if you wanna help here.

What other options do I have? I am willing to explore them. Anything to help get rid of this header.

You could trick jQuery into thinking it's always doing cross-domain requests:

$.ajaxSetup({
  crossDomain: true
});

// You probably want to make sure this won't
// break on browsers that do not support cors
$.support.cors = true;

But it's tricky because it will make all of the script and jsonp requests use script tag injection as a side effect.

Another solution is to extract the xhr transport from the source here, and do something like this:

$.ajaxTransport( "+", function( s ) {
   // your tweaked transport code here
});

That way, you can tweak your transport, making sure it is used instead of the default one (that's what the "+" specifies). A good way to get advantage of the Google CDN while customizing the internals of ajax to suit your needs (at the cost of some more bytes upfront though but nothing like the entire jQuery script).

Hopefully I have explained why I felt the need to ask for this. For 2% it was worth the 5 minutes required to write the initial ticket. It has now spiralled into a much bigger use of my time and yours (for which I am grateful, even if that's difficult to express in a debate like this).

That's much appreciated here too ;)

Let me summarize what are the blockers our side:

  • we don't want of an option specific to X-Requested-With
  • we don't want to give the impression you can remove headers set by the browser itself (header: { Origin: false } if you see what I mean), that would be a hugely confusing API (works for some headers, not for others)
  • we don't want nasty breakage because some third-party script is trigger happy

On the other hand, it would be nice not to have X-Requested-With set by default. It should probably have been on an opt-in basis from the start :/

We could remove the default X-Requested-With value knowing that it would be dead easy to add it back manually for all requests:

$.ajaxSetup({
  headers: {
    "X-Requested-With": "XMLHttpRequest"
  }
});

However, this would pose a problem for cross-domain requests since we would have to special case this specific header in the transport anyway which kinda defeats the purpose of not handling it anymore. Which brings us back to the idea of being able to unset headers (so that you could unset X-Requested-With for your crossDomain requests).

That's why it's probably better if we act as if X-Requested-With was a standard header set by the browser (nearly like what we do know, if we except it's possible to override it with another non-empty value). After all, there are (bad) tricks to remove it anyway (as given earlier in this comment).

So, tl;dr: it doesn't seem a widespread enough need to warrant the breakage and confusion needed to make it possible without resorting to existing tricks.

Please don't try to read too much into my choice of words earlier. I report bugs in software only when I have reached the end of a frustrating road trying to get the software to do as I wish it to (jQuery being the latest in a *very* long list). Thus I am already annoyed/angry/upset/pissed off by the time I click the "New Ticket" button and vent into a text field. My all-time worst job would be to work on the customer support phones for an ISP or utility company and have to deal with angry customers day in, day out!

I guarantee that answering bug reports and feature requests on a project like jQuery is quite the jading experience sometimes. We know about frustrated customers! ;P

Please, let us know how this went and what your conclusions were regarding the incidence of packet losses on those timeouts. You made me very curious.

comment:7 Changed 7 years ago by dcherman

FWIW just because I found this interesting, you could probably do this with the least amount of additional code ( and avoiding the crossdomain issues ) by removing the capability for setting that header entirely from the XHR if you absolutely never want it set.

https://gist.github.com/dcherman/5054258

If you only want to do it for requests to your domain, then you could use the same concept but apply it in a prefilter and restrict the conditions when it's done.

Note: See TracTickets for help on using tickets.