need some advice on x y plot

[nephish]

Hey there,
i have tried about every graphing package for python i can get to work
on my system. gnuplot, pychart, biggles, gdchart, etc.. (cant get
matplot to work)
so far, they all are working ok. I need to make an x y chart for some
data that comes in from sensors at different times durring the day. i
need it to show the value in the y and the time in the x . no problem
so far. But what i cannot get to happen is to scale x (time of the
plot) with respect to time. in other words, i get a chart with the
times evenly spaced out along the x axis, with their respective values.
i need the chart to show gaps when there are gaps in the data. i need
it to be able to scale by time. if i have 3 values that come in within
a few minutes, i need them to be displayed close together, as compared
to another value that may come in, say, an hour later. Does this make
sence ?
one of you guys know a charting app that will do this ? or is there
some other way i could do it?

looking for suggestions,
sk

 

[Grant Edwards]

[...] I need to make an x y chart for some data that comes in
from sensors at different times durring the day. i need it to
show the value in the y and the time in the x . no problem so
far. But what i cannot get to happen is to scale x (time of
the plot) with respect to time.

Gnuplot does that just fine. Just give it two columns of data, the
first being the x value (time) and the second being the y value.

All of the other plotting packages handle this as well.

 

[nephish]

how ?
i have tried to use unix timestamps, and i have also tried with
DateTime objects
do i need to use a scale that isn't linear (default in most) ?
how do i putt this off ?

thanks btw.
sk

 

[Larry Bates]

I would try to live with time scale being fixed and insert
None (or whatever value is used by charting package) for
times where observations were not taken. This will mean that
you have to preprocess your data by determining a time step
step value that will fit your data. If you get 3 observations
each 10 minutes apart then one an hour later, you will need to
insert 5 empty observations in the list before the last
observation.

Example:

1:00 <value1>
1:10 <value2>
1:20 <value3>
2:20 <value4>

1:00 <value1>
1:10 <value2>
1:20 <value3>
1:30 <empty value>
1:40 <empty value>
1:50 <empty value>
2:00 <empty value>
2:10 <empty value>
2:20 <value4>

Maybe this will be of some help and I'm sure there are other ways.

-Larry

 

[nephish]

i have thought about doing this, just a little different. i was going
to list the value pairs.
take the start time and end time and plot 100 empty plots between them.
add the value pairs, sort by time, and then draw it. The only thing is
it get kinda complicated when the times change a lot. they could span
an hour, a month, anything in between.

i like your idea, i will check various packages for how to plot 'none'

thanks
sk.

 

[Grant Edwards]

i have tried to use unix timestamps,

That has always worked for me. What happened?

and i have also tried with DateTime objects

Never tried that.

do i need to use a scale that isn't linear (default in most) ?
No.

how do i putt this off ?

Huh?

Gnuplot by default does exactly what you seem to want if you
just pass it x,y values.

 

[Grant Edwards]

I would try to live with time scale being fixed

I don't understand what you mean by "the time scale being
fixed". It's not. If you just pass the time,value pairs to
gnuplot, it does exactly what it should.

and insert
None (or whatever value is used by charting package) for
times where observations were not taken. This will mean that
you have to preprocess your data by determining a time step
step value that will fit your data. If you get 3 observations
each 10 minutes apart then one an hour later, you will need to
insert 5 empty observations in the list before the last
observation.

Example:

1:00 <value1>
1:10 <value2>
1:20 <value3>
2:20 <value4>

1:00 <value1>
1:10 <value2>
1:20 <value3>
1:30 <empty value>
1:40 <empty value>
1:50 <empty value>
2:00 <empty value>
2:10 <empty value>
2:20 <value4>

That's completely unnecessary. Just pass a set of time,value
pairs and they'll get plotted as desired.

 

[Grant Edwards]

and insert None (or whatever value is used by charting
package) for times where observations were not taken. This
will mean that you have to preprocess your data by determining
a time step step value that will fit your data. If you get 3
observations each 10 minutes apart then one an hour later, you
will need to insert 5 empty observations in the list before
the last observation.

[...]

That's completely unnecessary. Just pass a set of time,value
pairs and they'll get plotted as desired.

For example, here's how gnuplot plots the data

2 12
3 10
4 9
101 8
102 6
103 9



20 ++----------+-----------+-----------+----------+-----------+----------++
+ + + + + "foo.dat" A +
| |
| |
| |
15 ++ ++
| |
| |
|A |
| |
10 ++A ++
| A A |
| A |
| |
| A |
5 ++ ++
| |
| |
| |
+ + + + + + +
0 ++----------+-----------+-----------+----------+-----------+----------++
0 20 40 60 80 100 120


Isn't that what you're asking for?

 

[Kent Johnson]

how ?
i have tried to use unix timestamps, and i have also tried with
DateTime objects
do i need to use a scale that isn't linear (default in most) ?
how do i putt this off ?

Here is some code that works for me. It plots multiple datasets against time. The input data looks like this:
2005-04-04 16:00:00 141.154.195.129 - W3SVC1 SP6018ASP2 208.254.37.191 443 GET /rkadqsr/newskills/newskills.cfm selectedTab=1 200 0 53440 599 1594 HTTP/1.1 adqsr.skillport.com Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1) CookiesEnabled=1;+ASPSESSIONIDACTSSRAT=MCNLNLGADPOFGFKAHJHLDDKG;+CFID=785030;+CFTOKEN=27203160 https://adqsr.skillport.com/rkadqsr/login/login.cfm
2005-04-04 16:00:00 208.254.38.216 - W3SVC1 SP6018ASP2 208.254.37.191 443 POST /_rkadqsrBackend_od_cgi/aiccisapi.dll - 200 0 372 274 4547 HTTP/1.0 adqsr.skillport.com aiccisapi - -
2005-04-04 16:00:00 208.254.38.240 - W3SVC1 SP6018ASP2 208.254.37.191 443 POST /_rkadqsrBackend_od_cgi/aiccisapi.dll - 200 0 372 3019 250 HTTP/1.0 adqsr.skillport.com aiccisapi - -

import datetime, time
import Gnuplot

dataPath = 'ex05040416.log'

datasets = {}
startTime = None

f = open(dataPath)

for line in f:
try:
dat, tim, c_ip, cs_username, s_sitename, s_computername, s_ip, s_port, \
cs_method, cs_uri_stem, cs_uri_query, sc_status, sc_win32_status, sc_bytes, \
cs_bytes, time_taken, cs_version, cs_host, cs_User_Agent, cs_Cookie, cs_Referer = line.split()
except ValueError:
print "Can't parse", line
continue

tim = time.mktime(time.strptime(dat+' '+tim, "%Y-%m-%d %H:%M:%S"))
delay = int(time_taken)

if startTime is None:
startTime = tim

tim -= startTime

# print tim, delay
datasets.setdefault(sc_status, []).append([tim, delay])

g = Gnuplot.Gnuplot(debug=1)

plotter = g.plot
for key, values in datasets.items():
plotter(values)
plotter = g.replot

raw_input('Please press return to continue...\n')

 

[nephish]

ok, yeah, thats exactly what i am looking for. i will give it a go.
thanks a whole lot.

putt this off is a typo, pull this off is what i was meaning to type.

this is cool.

 

[nephish]

ok, i tried something similar to what you posted.
a little simpler though.
i am running a query on a database and making a list of time, value
pairs
kinda like this
plot_points = ([time, value], [time, value], [time, value])
gnuplot complains that it needs a float for one of the values.
i can plot just the value, and it shows up ( no x value found)

how should i proceed?

 

[Kent Johnson]

i am running a query on a database and making a list of time, value
pairs
kinda like this
plot_points = ([time, value], [time, value], [time, value])
gnuplot complains that it needs a float for one of the values.
i can plot just the value, and it shows up ( no x value found)

how should i proceed?

Convert one of the values to a float? What are your time and value numbers?

Kent

 

[nephish]

the time is DateTime.DateTime object from a mySQLdb query.
the value is a number anywhere between 0 and 15.
the datetime is formatted like 2005-10-20 08:40:34

i could strip it and make a timestamp out of it. but reading the
number of seconds since january of 1970 doesn't make a neat chart.

any suggestions?

oh, and can you output from Gnuplot to a png or jpeg or something like
that.
the Gnuplot documentation says that it can make a bitmap png but i dont
know how to do that with the python wrapper. the docs for this are a
little cryptic.
still a newbie here. if you know a good tutorial on this, i would
really appreciate the link.

thanks for your time, and brain

 

[William Park]

the time is DateTime.DateTime object from a mySQLdb query.
the value is a number anywhere between 0 and 15.
the datetime is formatted like 2005-10-20 08:40:34

i could strip it and make a timestamp out of it. but reading the
number of seconds since january of 1970 doesn't make a neat chart.

any suggestions?

Python is capable of integer arithmetic, eg. 1500

--
William Park <[email protected]>, Toronto, Canada
ThinFlash: Linux thin-client on USB key (flash) drive
http://home.eol.ca/~parkw/thinflash.html
BashDiff: Super Bash shell
http://freshmeat.net/projects/bashdiff/

 

[Grant Edwards]

the time is DateTime.DateTime object from a mySQLdb query. the
value is a number anywhere between 0 and 15. the datetime is
formatted like 2005-10-20 08:40:34

i could strip it and make a timestamp out of it. but reading
the number of seconds since january of 1970 doesn't make a
neat chart.

Pass the data values as floating point numbers (I typically use
seconds since the test run started). If you the x tics to be
something else, then change them. It's pretty simple to label
the x-axis using normal date/time notation.

any suggestions?

oh, and can you output from Gnuplot to a png or jpeg or
something like that.

Is that a question? If so, the answer is yes.

the Gnuplot documentation says that it can make a bitmap png
but i dont know how to do that with the python wrapper. the
docs for this are a little cryptic. still a newbie here. if
you know a good tutorial on this, i would really appreciate
the link.

Once you start doing "odd" stuff (e.g. custom labels for the x-axis,
output other than X11 or postscript), it's usually simplest to
just send normal gnuplot commands.

------------------------------8<------------------------------
import Gnuplot,time,sys,math

def pause():
sys.stdout.write("Press enter to continue: ")
sys.stdin.readline()

def fgrid(start,stop,count):
for i in xrange(count):
yield start + ((stop-start)*i)/count

start = time.time()

xdata = [x for x in fgrid(0,600.0,10)] # two minutes worth
ydata = [math.sin(x/100.0) for x in xdata]

data = Gnuplot.Data(xdata,ydata,with='linespoints')

gp = Gnuplot.Gnuplot()
gp.title('Data starting at %s' % time.ctime(start+xdata[0]))

# x axis will use default tics (seconds since start of run)
gp.plot(data)
pause()

# now let's change x tics to time of day in HH:MM:SS with
# ticmarks at 120 second intervals

ti = 120.0
startTic = int((start+xdata[0])//ti * ti)
endTic = int((start+xdata[-1])//ti * ti)
ti = int(ti)

ticx = range(startTic,endTic+ti,ti)
tics = [(x, time.strftime('%H:%M:%S',time.localtime(x))) for x in ticx]
ticstrings = ['"%s" %f' % (t[1],t[0]-start) for t in tics]

gp('set xtics (%s)' % ','.join(ticstrings))
gp.plot(data)
pause()

outfile = 'foo.png'
gp('set term png')
gp('set out "%s"' % outfile)
gp.plot(data)
------------------------------8<------------------------------

 

[nephish]

ok, i have a display, and its a work in progress.

lemme get this straight. you used

gp('set term png')

is this an example of sending normal gnuplot commands?
if so, are all of the gnuplot commands available ?
thanks so much this is helping me out a lot here

 

[Grant Edwards]

gp('set term png')

is this an example of sending normal gnuplot commands?
Yes.

if so, are all of the gnuplot commands available ?

Yes.

 

[nephish]

this is great, because the docs on gnuplots website are a bit easier
for me to grasp.
thanks so much for your time on this one. you really have helped me a
lot.
i will not get a change to work on this till monday. so i may have more
questions then. This is sure a point in the right direction.

thanks much.
shawn

 

[nephish]

ok, i am stuck a bit here with this line

gp('set xtics (%s)' % ','.join(ticstrings))

the error code says that it is looking for a closing parenthesis.
that and i am kinda new enough not to really get what %s is all about.
i know it formats a string.

can you simply pass a list to 'set xtics' ?

i mean, i got this from the gnuplot site

Syntax:

set xtics {axis | border} {{no}mirror} {{no}rotate {by <ang>}}
{offset <offset> | nooffset}
{ autofreq
| <incr>
| <start>, <incr> {,<end>}
| ({"<label>"} <pos> {<level>} {,{"<label>"}...) }
{ font "name{,<size>}" }
{ textcolor <colorspec> }
unset xtics
show xtics

but it does not look like all the options are necessary from the docs,
i guess i am asking, if i just wanted to pass a list of say, 14 tics to
the x axis as a list, what of the above is necessary ?

i built the list by taking time stamps in seconds (the same as being
plotted for x) took the end time minus the start time and divided by
12, incremented each by this amount until i had 12 plots (plus of
course the first and last).
these are all stored in plot_times[]..

any tips ?

sorry if this all sounds a bit scrambled. i just got the hang of
changing from a datetime.datetime to time in epoch seconds. ( these
little triumphs keep me going )

thanks for your help. the graph i built looks great, and looks right
referenced with time. just neeed to print the x a little easier to
read.

thanks again,

shawn

 

[Dennis Lee Bieber]

ok, i am stuck a bit here with this line

gp('set xtics (%s)' % ','.join(ticstrings))

the error code says that it is looking for a closing parenthesis.
that and i am kinda new enough not to really get what %s is all about.
i know it formats a string.

can you simply pass a list to 'set xtics' ?

i mean, i got this from the gnuplot site

Syntax:

set xtics {axis | border} {{no}mirror} {{no}rotate {by <ang>}}
{offset <offset> | nooffset}
{ autofreq
| <incr>
| <start>, <incr> {,<end>}
| ({"<label>"} <pos> {<level>} {,{"<label>"}...) }
{ font "name{,<size>}" }
{ textcolor <colorspec> }
unset xtics
show xtics

The only option I see that uses parens and commas is the fourth
option in the block starting { autofreq.

The problem is: What does ticstrings contain?
--