Is load testing a web service with seperate threads from one machinethe same as a single thread from

[ses]

Well the title says it all really, I was wondering if testing a web
service for the load it can take in terms of simultaneous requests by
using multiple threads from one machine is essentially the same /
emulates the conditions as testing it from separate machines using one
thread per machine?

I think obviously there is some difference, but fundamentally are you
essentially testing the ability of something to scale by doing
multiple threads from the same machine?

Thanks.

 

[Lew]

Well the title says it all really, I was wondering if testing a web
service for the load it can take in terms of simultaneous requests by
using multiple threads from one machine is essentially the same /
emulates the conditions as testing it from separate machines using one
thread per machine?

I think obviously there is some difference, but fundamentally are you
essentially testing the ability of something to scale by doing
multiple threads from the same machine?

What does "essentiallY" mean?

Can one machine generate the load profile you want to use for your test? What
happens when you try?

 

[Esmond Pitt]

The first thing you will discover is that multiple logical clients on a
single physical host will exhaust the port space with outbound
connections and start incurring BindExceptions. This doesn't happen with
multiple physical client hosts. So you may not even be able to get to
the required number of simultaneous connections. Windows limits this
space to a few thousand.

 

[ses]

The first thing you will discover is that multiple logical clients on a
single physical host will exhaust the port space with outbound
connections and start incurring BindExceptions. This doesn't happen with
multiple physical client hosts. So you may not even be able to get to
the required number of simultaneous connections. Windows limits this
space to a few thousand.

Thanks for the insight, a few thousand connections should do for me,
in fact if I could test at least 100 simultaneous connections that
would be suitable for now. I suppose the take home message is to try
it and see what the load looks like by logging from the web service
itself.

Out of interest would it be any different if I used a server with a
number of virtual machines? (Not that I really have that level of
resources available to me!)

 

[Esmond Pitt]

Out of interest would it be any different if I used a server with a
number of virtual machines? (Not that I really have that level of
resources available to me!)

*Everything* that represents a difference between the testing
environment and the deployment environment is significant. In this case
you would testing the server with 1/N less CPU power than would be
available if there were N physical rather than virtual servers. Prety
significant, don't you think?

 

[Lew]

*Everything* that represents a difference between the testing environment and
the deployment environment is significant. In this case you would testing the
server with 1/N less CPU power than would be available if there were N
physical rather than virtual servers. Prety significant, don't you think?

Real-world anecdote: I was on a big-iron project a few years ago where the
test environment and production environment servers were two different models,
with differing core counts, of the same mainframe server brand in the same
line (the test environment's was the little brother of the production server),
and that made it impossible for the test environment to replicate certain
conditions that happened in production.

There are formal means to project probabilistically from test results in such
situations, but in practice those who know how to do so rarely get the
authority to design the test plans.

Then again, such people are rare.