XmlHttpRequest not loading latest version of xml

[kwilder]

This works, but it doesn't load the latest version of the xml if it was
just modified without closing and reopening the browser.

Here's the scenario:

I have an xml doc called results.xml. It can contain lots of data.

<Results>
<Data>Stuff</Data>
<Data>More stuff</Data>
</Results>

My ASP application can add to the data during the year. At the end of
the year I want to reset the xml.

I run a script that can make a copy of the file from results.xml to
results_[yyyymmdd].xml as a backup and save a new results.xml file as
the following with a backup date attribute:

<Results BUDate="20051124"></Results>

So far this all works fine, but here's where the problem occurs. I've
set a Backup date attribute on the off chance I want to restore that
lastest file.

I have a checkbox that will toggle between enabled and disabled
depending on whether the results.xml file has an attribute "BUDate".
If I try to restore the backup immediately, during the same session it
returns the results.xml file as it was before it was backed up. In
other words, it has all the data!

In order to get it to work correctly, I have to log off, close the
browser (IE) and re-open the browser.

Is there any way to make it retrieve that latest xml version without
closing the browser?

Many thanks,

King Wilder


Here's my javascript code:

var response = null;

// global flag
var isIE = false;

// global request and XML document objects
var req;

function loadXMLDoc(url, GoAsync) {

// branch for native XMLHttpRequest object
if (window.XMLHttpRequest) {
req = new XMLHttpRequest();
req.onreadystatechange = processReqChange;
req.open("GET", url, GoAsync);
req.send(null);
// branch for IE/Windows ActiveX version
} else if (window.ActiveXObject) {
isIE = true;
req = new ActiveXObject("Microsoft.XMLHTTP");
if (req) {
req.onreadystatechange = processReqChange;
req.open("GET", url, GoAsync);
req.send();
}
}
}

// handle onreadystatechange event of req object
function processReqChange() {
// only if req shows "loaded"
if (req.readyState == 4) {
// only if "OK"
if (req.status == 200) {
response = req.responseXML;
} else {
alert("There was a problem retrieving the XML data:\n" +
req.statusText);
}
}
}

 

[Matt Kruse]

This works, but it doesn't load the latest version of the xml if it
was just modified without closing and reopening the browser.

GET requests can be cached by the browser, just as they could by typing the
url into the browser.
Attach a unique string to the url with each request so the browser doesn't
cache it.

You may want to consider using a wrapper around XMLHttpRequest to make it
easier to deal with and to automate handling of issues like this. For
example, my library at http://www.AjaxToolbox.com/ will automatically add a
unique ID to GET requests.

 

[Thomas 'PointedEars' Lahn]

GET requests can be cached by the browser, just as they could by typing
the url into the browser.

It is not the request that is cached but the resource retrieved, hence the
_response_ previously sent. Which is why GET and POST request make no
difference here.

Attach a unique string to the url with each request so the browser doesn't
cache it.

No, that Dirty Hack will only fill the user's cache with obsolete
data over time. <URL:http://www.mnot.net/cache_docs/>


PointedEars

 

[Matt Kruse]

It is not the request that is cached but the resource retrieved,
hence the _response_ previously sent. Which is why GET and POST
request make no difference here.

Semantics.

GET and POST differ because servers typically expire POST requests
immediately, so the user doesn't need to worry about anything.

No, that Dirty Hack will only fill the user's cache with obsolete
data over time. <URL:http://www.mnot.net/cache_docs/>

It's not a dirty hack - it's often the best option. Users can't always
control server headers, and often they would have no idea how to do it.
Creating unique url's won't "full the user's cache" any more than they allow
for, and no more than browsing around the web.

 

[Thomas 'PointedEars' Lahn]

Semantics.

Certainly not. What may be cached locally here is the server's response
to the client's request and the URI assigned with that request so that
the response can be identified without submitting the request again. It
is definitely not the request itself that is cached here across sessions
(as that would pose a security risk).

GET and POST differ because servers typically expire POST requests
immediately,

First of all, (in contrast to FTP, for example) HTTP is not a stateful
protocol by default. There is only one HTTP request and one HTTP response
to it. The very notion of "typically expire POST requests" regarding HTTP
servers is absurd; if one would follow this logic, one would be forced to
state that HTTP servers "expire" _every_ HTTP request which is obviously
nonsense. That said, second, when a _connection_ expires (since several
requests can be issued and responded to using the same connection) depends
on whether the client is sending Keep-Alive or not, specified since
HTTP/1.1. (You probably want to read RFC2616, especially section 1.4.)
However, that has nothing to do with the matter discussed, since the
latter is not a server-side problem (although it may seem so), but a
client-side one that can be helped best with proper server-side
configuration.

True is that HTTP servers can sustain a cache for paths of requested
resources and responses to these requests. However, in Apache HTTPD,
the most advanced and widest distributed of HTTP servers, such caching
is still experimental and only present since version 2.0 (where 1.x
is the most version used for production environments.)[1]
Another widely distributed HTTP server is Microsoft Internet Information
Server. "By default InetInfo, the process responsible for WWW, FTP and
Gopher uses a 3MB of cache for all of the services."[2] That has perhaps
changed. See also [3] and please CMIIW.

True is that HTTP _proxy_ servers cache requests and responses and they
are less likely to cache POST requests and responses to them. However,
that is a _different_ matter. Using a proper caching technique on the
target HTTP server helps intermediary proxy servers to identify and hold
only the latest version of the resource, though, where in contrast a
different URI fills their cache with a new one and inevitably forces
other cached resources to be removed. Since a proxy server is used by
many people, it is not a very social thinking that initially caused this,
if you ask me.

True is that responses to POST requests are not likely to be kept by
caches as described in /Caching Tutorial for Web Authors and Webmasters/.
However, some clients do cache POST requests for the current session.
That is why e.g. in Firefox you can use the Back button/feature from a
response to a POST request and return to the resource that issued sending
it. You then can use the Forward button/feature and are asked if you
really want to re-submit the POST request. This works as long as the
history is sustained. However, either is _different_ from the matter
discussed.

so the user doesn't need to worry about anything.


It's not a dirty hack - it's often the best option.

It is _never_ the best option. A proper caching technique is always the
best option.

Users can't always control server headers, and often they would have
no idea how to do it.

Then they are incompetent enough not to be Web authors and programmers!

Creating unique url's won't "full the user's cache" any more than they
allow for, and no more than browsing around the web.

Without proper caching technique and especially with abusing the query-part
of URIs, content that is obsolete the moment it is retrieved is cached,
filling the caches with data that is useless (since unlikely to be accessed
again) for a long time, unless removed later by caching requests that would
exceed the assigned cache memory and disk space -- iff that really worked
locally -- or removed manually by the user (who wonders where his free disk
space is since the former does _not_ really work in one of the widest
distributed Web user agents).


PointedEars
___________
[1] <http://httpd.apache.org/docs/2.0/mod/mod_cache.html>
[2] <URL:http://www.windowsitpro.com/Article/ArticleID/13974/13974.html>
[3]
<URL:http://msdn.microsoft.com/library/en-us/iissdk/html/4f0f204a-eb78-4e15-b52b-89c6dbb343ef.asp>

 

[Jim Ley]

No, that Dirty Hack will only fill the user's cache with obsolete
data over time. <URL:http://www.mnot.net/cache_docs/>

Except it will be reliable in all situations, whereas suggesting that
a resource should not be cached won't.

You should certainly include headers suggesting it's not cached too,
to save the "fill the user's cache" problem you raise, but changing
the URI is the best idea.

Jim.

 

[Thomas 'PointedEars' Lahn]

[...] Thomas 'PointedEars' Lahn [...]

No, that Dirty Hack will only fill the user's cache with obsolete
data over time. <URL:http://www.mnot.net/cache_docs/>

Except it will be reliable in all situations,

Reliable -- perhaps. Reasonable and social -- no.

whereas suggesting that a resource should not be cached won't.

You should certainly include headers suggesting it's not cached too,

I am not suggesting such a thing.

to save the "fill the user's cache" problem you raise,

That will save from that problem but cause more server
and network load, burdening the problem on others. No.

The "trick" is to specify resources to expire and to make
different versions of them more identifyable for the client
when _served_.

but changing the URI is the best idea.

No, it is only the easiest one.


PointedEars

 

[Aaron Gray]

No, it is only the easiest one.

So its a server side problem. The headers the server serves should have some
form of "timeout".

Aaron

 

[Matt Kruse]

Certainly not. What may be cached locally here is the server's
response to the client's request and the URI assigned with that
request...

You're being argumentative as usual.
Clearly, it's the response that is cached by the browser. However, the
generic term 'request' is often used to refer to the whole request/response
cycle. Perhaps it is not the ideal term, but you have to be pretty obtuse to
argue over it.

It is _never_ the best option. A proper caching technique is always
the best option.

You're wrong.

Then they are incompetent enough not to be Web authors and
programmers!

Your elitist bullshit gets tiring.

 

[Michael Winter]

On 26/11/2005 17:15, Thomas 'PointedEars' Lahn wrote:

[Freshness information]

The response should also include such a header, yes. The specified
header name is Expires.

Or Cache-Control using the max-age directive (which takes precedence
where supported).

Other identifying headers include ETag and Content-Length, both
specified in HTTP/1.1.

The Content-Length header isn't used for entity validation. ETag is, and
so is Last-Modified. That said, the Content-Length header can otherwise
improve network performance as it's required for persistent connections
to be utilised.

Mike

 

[Thomas 'PointedEars' Lahn]

You're being argumentative as usual.

I do hope so since you provided no convincing arguments.

Clearly, it's the response that is cached by the browser. However,
the generic term 'request' is often used to refer to the whole
request/response cycle.

It is not, only in your parallel universe.

Perhaps it is not the ideal term, but you have to
be pretty obtuse to argue over it.

You have to pretty ignorant to ignore RFC2616.

You're wrong.

I am not.

Your elitist bullshit gets tiring.

So you _do_ prefer people who do not know what they are doing, so
that you can tell them what to do. "Who's more foolish? ..."


PointedEars

 

[Aaron Gray]

No, it is a client-side problem best to be solved server-side

Ah... I see

The response should also include such a header, yes. The specified
header name is Expires. Other identifying headers include ETag and
Content-Length, both specified in HTTP/1.1.

Thanks for the advice.

Seems like the best and only truely correct solution to the problem.

Aaron

 

[Thomas 'PointedEars' Lahn]

Or Cache-Control using the max-age directive (which takes
precedence where supported).

True (RFC2616, 14.9.3.)


PointedEars

 

[Randy Webb]

Thomas 'PointedEars' Lahn said the following on 11/26/2005 2:07 PM:

Matt Kruse wrote:




I do hope so since you provided no convincing arguments.

You are always argumentative, it takes no convincing arguments (or
proof) to know it. It only takes reading a few of your posts.

It is not, only in your parallel universe.

His "parallel universe" is called "Reality", your universe is called
"Theory". I prefer the former to the latter.

You have to pretty ignorant to ignore RFC2616.

Nah, it only takes a decent amount of common sense and a dose of reality
to ignore it. To blatantly think it is the end-all solution is the
ignorant part.

Theory: An RFC says to do something.
Reality: The browsers do something different.

That is something that you have failed to realize to date.

I am not.

Yes you are, you just don't realize it yet.

So you _do_ prefer people who do not know what they are doing, so
that you can tell them what to do. "Who's more foolish? ..."

Nah, he is like me. He prefers reality to elitist bullshit Theorists
like you.

 

[Nick]

In order to get it to work correctly, I have to log off, close the
browser (IE) and re-open the browser.

Is there any way to make it retrieve that latest xml version without
closing the browser?

It's simple enough to set the headers in server-side scripts, but doing
it for static files needs server access usually. I'd suggest wrapping
the request for the file in a script? I know nothing about ASP but I
know it's easy enough in PHP with something like

<?php
header("Cache-Control: no-cache, must-revalidate"); // HTTP/1.1
header("Expires: Mon, 26 Jul 1997 05:00:00 GMT"); // Date in the past
?>

and then get the script to output the file (either hardcoded or specify
somehow in GET).

Nick

 

[Thomas 'PointedEars' Lahn]

It's simple enough to set the headers in server-side scripts, but doing
it for static files needs server access usually. I'd suggest wrapping
the request for the file in a script? I know nothing about ASP but I
know it's easy enough in PHP with something like

<?php
header("Cache-Control: no-cache, must-revalidate"); // HTTP/1.1
header("Expires: Mon, 26 Jul 1997 05:00:00 GMT"); // Date in the past
?>

Note that this will create unnecessary traffic along the request/response
chain and slows down refreshs. For all practical purposes, Last-Modified
(which HTTP/1.1 servers SHOULD send and HTTP/1.1 caching proxies MUST NOT
handle as a cache[able] request) and a reasonable expiration date in the
not-too-far future (after which HTTP/1.0 implementations MUST NOT cache
the resource) is better. See also the ETag header which 'provides for
an "opaque" cache validator' as it "MUST be unique across all versions
of all entities associated with a particular resource (RFC2616, 13.3.2
and 3.11).

and then get the script to output the file (either hardcoded or specify
somehow in GET).

Either this way or the simpler one: Neither in ASP nor in PHP it is
necessary to use an additional file. Both ASP and PHP code to generate the
required HTTP headers can be included right into the resource requested, at
the very beginning of the file [in PHP, header() only works if there was no
previous output.]


PointedEars

 

[Nick]

Note that this will create unnecessary traffic along the request/response
chain and slows down refreshs.

I'm not a big expert on HTTP, I just copied that from the php.net
documentation on the header() function. You'd think they'd know better

Either this way or the simpler one: Neither in ASP nor in PHP it is
necessary to use an additional file. Both ASP and PHP code to generate the
required HTTP headers can be included right into the resource requested

If the resource requested is a dynamic page, yes. I believed the OP
wishes to request a static XML file though.

 

[1.mRW1Zard.1]

Pointed Ears may have a good amount of knowledge, and maybe a good
amount of experience with development with the XmlHTTPRequest Object,
but what he is pushing is his knowledge, but not practical experience.

Others have been expressing that practical experience with this object
does have responses being cached in the browser, this is for a variety
of reasons:

1. Server is not configured to expire content
2. Client Browser is just holding on to the cached version of the
response.

To name two reasons.

The truth is, that no matter how much knowledge you have if you do not
expect the worse case with your applications it is doomed to faile.
Take this example, I build a web application that runs in IE and I use
XmlHTTPRequest to get me data so that I can update my web interface,
but I am using a service that I did not build and I have no control
over. They are not going to change their server settings for just my
one application and I need to use this so that I can get my application
running. So I set my headers and test out hitting the service, but end
up always getting back the same information, I then add the Unique ID
and get fresh data each time I make the request. This is a real world
example of using all the knownledge being thrown around, and exists in
many applications today. Now if you were to create your own data
channel that the XmlHTTPRequest Object is to use, then you make the
header changes on the server and test the application calling it with
out the Unique ID, if this works by giving you freash data every time,
then you have your solution but if you find it buggy and it isn't 100%
reliable you must add the extra over head of a Unique ID to make it
that way.

The application is what takes priority not the theory of what should
work and how it should work. The web was built by people with all
levels of knowledge and personalities. Some do not even want to change
there data services others are more than willing to make a change as
long as that they see it benifical.

 

[Thomas 'PointedEars' Lahn]

Pointed Ears may have a good amount of knowledge, and maybe a good
amount of experience with development with the XmlHTTPRequest Object,
but what he is pushing is his knowledge, but not practical experience.
Wrong.

[...]

Your arguments have already been refuted, I suggest that you read more
carefully.


PointedEars

 

[1.mRW1Zard.1]

I had read them, and stated these as what are in place today and what
is being developed with currently. You have not stated any practical
examples that you had created that the topic was asking for, again all
you had stated was theory. Give me an example of your theory and I
will take back my statement of you just pushing knowledge.

I am not doubting your understanding of the subject, but I do doubt
that you have not built many applications that use this and consume
many services, ones that you control and others that you do not
control.

I can be refuted as much as anyone wants, it doesn't stop that fact
that of what is in place today is what works and what has been proven
to work. It may not be perfect, but it is what it is because software
is not perfect because has been made by practical developers not
theologians.