building a web interface

[Shel]

Hello,

I am pretty new to all this. I have some coding experience, and am
currently most comfortable with Python. I also have database design
experience with MS Access, and have just created my first mySQL db.

So right now I have a mySQL db structure and some Python code. My end
goal is to create a browser-based interactive fiction/game thing. My
code is currently just using dummy data rather than pulling in data
from the db, but I think/hope it won't be too big of a deal to
interact with the db through Python (famous last words...).

My main problem right now is how to do the web interface. I don't know
much about web architecture, unfortunately. I think I need a CGI
script?

What I would really like is to find a GUI tool to design the interface
that would have customizable templates or drag-and-drop elements or
something, so I wouldn't have to code much by hand. Something easy, if
such a tool exists.

Also would prefer something that I would be able to install/use
without having much access to the server where the site will be hosted
(on a university server that I don't have control over). I don't fully
understand how a lot of the tools I have been looking at work, but it
seems like they're often things where you'd have to get an
administrator to do stuff on the server (is that right?), which I
would prefer to avoid.

It looks like Django has some sort of standalone implementation, but
I'm not clear on how that would work or how much of a learning curve
there would be for using it.

If anyone has any thoughts or suggestions, I would really appreciate
it.

Hope my questions make sense. I don't really know what I'm doing, so
could be they're a bit silly. I apologize if that's the case, and
please let me know if you need any additional informmation.

Thanks,
Shel

 

[Martin Gregorie]

Hello,

I am pretty new to all this. I have some coding experience, and am
currently most comfortable with Python. I also have database design
experience with MS Access, and have just created my first mySQL db.

So right now I have a mySQL db structure and some Python code. My end
goal is to create a browser-based interactive fiction/game thing. My
code is currently just using dummy data rather than pulling in data from
the db, but I think/hope it won't be too big of a deal to interact with
the db through Python (famous last words...).

Wrong approach. If you're going to use a database, start with writing a
program that puts your data into it alongside the program that reads and
uses the data. Do it incrementally: if you are writing an adventure game
a good place to start would be the 'place' table and the part of the game
builder that adds/amends/removes a place from the database as well as
enough of the game program to show the place description and use the
doors to move from place to place. This way you'll have that working
before you start adding weapons, monsters, etc. or try to implement
fights. You'll also learn that writing place descriptions isn't as easy
as it sounds: if there are several doors into a room the description must
make sense no matter which way you come into it. Its quite hard at first
to avoid descriptions like "Further into the forest. Its getting darker"
which may be OK coming from a road but is pretty stupid if you're on your
way out of the forest toward the road.

http://www.kitebird.com/articles/pydbapi.html which tells you what you
need to know about accessing SQL databases.

My main problem right now is how to do the web interface. I don't know
much about web architecture, unfortunately. I think I need a CGI script?

You're biting off quite a lot at once. It may be easier to first write a
command line Python program and get it working in that form. Once you
have the game/application logic working you can adapt it to run as a CGI
script. Development and debugging will be easier that way.

If you're not familiar with this style of adventure game, look here:
http://www.rickadams.org/adventure/ You'll find various downloadable
examples on the downloads page as DOS executables and in C or Fortran
source code. Have a play with it to see what even this very basic user
interaction can do - you'll have fun and learn useful stuff.

What I would really like is to find a GUI tool to design the interface
that would have customizable templates or drag-and-drop elements or
something, so I wouldn't have to code much by hand. Something easy, if
such a tool exists.

If you're doing a textual adventure of something like it a tool won't
gain you much, since you only need a few skeleton web pages than your
program can select and drop text or images into.

Also would prefer something that I would be able to install/use without
having much access to the server where the site will be hosted (on a
university server that I don't have control over). I don't fully
understand how a lot of the tools I have been looking at work, but it
seems like they're often things where you'd have to get an administrator
to do stuff on the server (is that right?), which I would prefer to
avoid.

CGI and Python should be OK providing the server has Python available and
the admins will let you use CGI, but that's not a good development
platform.

You really need a web server you can start, stop and fiddle with once you
get to the point of putting your code in a server. Running a simple local
web server on your computer would be a better place to start: once its
running you simply point your browser at localhost:80 and send it the URL
of the initial page of your application.

Search on "Python web server" for details of building or downloading a
simple one. You'll also find stuff about interfacing Python programs to a
web server.

 

[Dennis Lee Bieber]

What I would really like is to find a GUI tool to design the interface
that would have customizable templates or drag-and-drop elements or
something, so I wouldn't have to code much by hand. Something easy, if
such a tool exists.

In original pure HTML, such would be rather meaningless... Pure HTML
leaves the presentation (the rendering) of the page up to the user's
browser. Layout of one item against another is only relative (form field
2 comes after form field 1). And I've seen some rather ugly pages
produced using so-called graphical design tools (like layers of
<i><b><i><b> text </b></i></b></i> because the tool isn't smart enough
to find and remove prior tag pairs when one changes intent... I've also
seen <i><b> text </i></b>).

Frames were one attempt at dividing a browser into regions in which
some stuff could stay constant while others are scrolled.

Then you get the hoard that creates things like transparent GIF
files with a few pixels to create "spacing" elements...

I tend to use an ancient copy of HomeSite 5 for HTML editing --
though I don't do enough such to justify taking the time to define
things like Django's templating tags as standard types. I prefer to edit
in the original intent of HTML -- as a mark-up language which identifies
WHAT a construct is, but does not bother with HOW/WHERE that construct
is rendered. If I really need something looking more like a newspaper
layout I'd define a table and use combinations of row and column span to
"grid" the contents as desired -- and even that would still use
proportional spacing (90% of browser width, say) rather than absolute
pixels (845 pixels wide).

Also would prefer something that I would be able to install/use
without having much access to the server where the site will be hosted
(on a university server that I don't have control over). I don't fully
understand how a lot of the tools I have been looking at work, but it
seems like they're often things where you'd have to get an
administrator to do stuff on the server (is that right?), which I
would prefer to avoid.

First step -- find out what toolset the server uses; Apache with
mod-python? mod-WSGI? Do they even have Python available? and if so,
what version (do they have MySQL available AND MySQLdb, and how much
control will they give you -- MySQL administration could be set up to
where they have to create the database AND table definitions, and all
you get is read/update/delete data privileges... Or they create the
database with no contents but your account has table manipulation
privileges within that database... Can you administer your database vie
direct login to the MySQL server, or do you have to go through some
web-based interface?).

It looks like Django has some sort of standalone implementation, but
I'm not clear on how that would work or how much of a learning curve
there would be for using it.

The "standalone" is a development/debug server mode running on a
non-standard port number (HTTP is privileged port 80, these types of
development servers often run on ports 8080, 8081, etc. as one does not
need to be superuser to bind to the port number). It is not meant as a
production server. Instead one has the real server configured to run the
application (Django and similar are not simple stand-alone CGI scripts;
pages are _methods/functions_ within the application which generate the
page contents and return them to the server for delivery).

 

[Shel]

X-No-Archive:Thanks for your advice. Will take a look at
rickadamsadventure.org. Although I am not actually creating an
adventure game, I'm sure it will be helpful.

Sorry I wasn't clear about the db part. Most of the data has already
been written and/or generated and tested with the code. I do plan to
finish all the database stuff before going to the front end, but am
just thinking ahead about how to do the interface.

 

[Shel]

Thanks for your help.

I know that they have Python and mySQL. They have mySQL set up so
that they just have to create the db, which they've already done and
I've defined the tables and modified the data and whatnot, so that
seems good. Not sure about mod-python v. mod-wsgi.

Ah, the Django thing makes more sense to me now. I couldn't quite
figure out what the heck they were talking about in the description of
it.

Shel

 

[Shel]

Thanks for your help.

I know that they have Python and mySQL. They have mySQL set up so
that they just have to create the db, which they've already done and
I've defined the tables and modified the data and whatnot, so that
seems good. Not sure about mod-python v. mod-wsgi.

Ah, the Django thing makes more sense to me now. I couldn't quite
figure out what the heck they were talking about in the description of
it.

Shel

 

[Shel]

This looks very promising. Thanks so much!
Shel

you might take a look at web2py,
it can be handled quit low level,
runs perfectly on a the build python server,
and you switch to almost any database at any time

cheers,
Stef

 

[Ian Kelly]

So right now I have a mySQL db structure and some Python code. My end
goal is to create a browser-based interactive fiction/game thing. My
code is currently just using dummy data rather than pulling in data
from the db, but I think/hope it won't be too big of a deal to
interact with the db through Python (famous last words...).

Suggestion: unless you're intent on reinventing the wheel, why not just
set up your web interface as a thin front-end for an existing IF engine?
An excellent starting point for this would be digging up the source
for InfocomBot [1], an AIM bot that acts as a simple wrapper around
Frotz [2]. My recollection of seeing the source once is that it was
only around 50 or so lines of code. Doing it this way,

1) You don't have to write an IF engine from scratch.

2) If you wrap Frotz specifically, you can serve any Z-code game ever
written, including (I think) all of the Infocom games.

3) You can create your game using powerful existing development tools,
such as Inform [3].

Cheers,
Ian

[1] http://waxy.org/2004/03/infocombot_for/
[2] http://frotz.sourceforge.net/
[3] http://inform7.com/

 

[Martin Gregorie]

Sorry I wasn't clear about the db part. Most of the data has already
been written and/or generated and tested with the code. I do plan to
finish all the database stuff before going to the front end, but am just
thinking ahead about how to do the interface.

That sounds good. Sorry if I was repeating stuff you already know, but it
wasn't obvious what you knew about care & feeding of an RDBMS. I'll just
add two comments on databases:
- Decompose the database design to 3NF form and make sure all prime
and foreign keys have indexes. This is stuff that previous experience
shows self-taught Access users don't do. Not doing it will bite you
hard on performance as soon as the tables exceed a few rows in size.
Fixing it later can force major changes to the programs as well.

- If you haven't looked at it yet, find out about the EXPLAIN verb
and what its output means. Use it on all queries that your online
program issues and take notice of how rearranging the query and/or
adding/changing indexes affects the cost of the query. Lower cost
queries mean higher performance and hence faster response times.

What I meant to add last night is that, if your application is to be used
by more than a single user at a time a prime consideration is how you
will recognise input received from each user and how you'll store their
context between interactions with them in the same session and keep each
session's context separate. The web server doesn't do this, so this
managing session context is the application's responsibility. Common
methods are to use a session cookie and/or to store session context in
the database.

 

[Shel]

Definitely not looking to reinvent the wheel. Will check these out,
thanks!
Shel

So right now I have a mySQL db structure and some Python code. My end
goal is to create a browser-based interactive fiction/game thing. My
code is currently just using dummy data rather than pulling in data
from the db, but I think/hope it won't be too big of a deal to
interact with the db through Python (famous last words...).

Suggestion: unless you're intent on reinventing the wheel, why not just
set up your web interface as a thin front-end for an existing IF engine?
  An excellent starting point for this would be digging up the source
for InfocomBot [1], an AIM bot that acts as a simple wrapper around
Frotz [2].  My recollection of seeing the source once is that it was
only around 50 or so lines of code.  Doing it this way,

1) You don't have to write an IF engine from scratch.

2) If you wrap Frotz specifically, you can serve any Z-code game ever
written, including (I think) all of the Infocom games.

3) You can create your game using powerful existing development tools,
such as Inform [3].

Cheers,
Ian

[1]http://waxy.org/2004/03/infocombot_for/
[2]http://frotz.sourceforge.net/
[3]http://inform7.com/

 

[Shel]

No worries at all about repeating things. I wasn't clear, and I
appreciate your going to the trouble of teaching me just about
anything. Even things I think I know, I might not really know

Let's see... am okay with the relational design stuff. Thanks for the
"EXPLAIN" verb.

I am confused about multiple simultaneous users, which I would like to
be able to accommodate. On the db side, I have a structure to store
data for each user, and know a bit about selectively locking data,
although I have not implemented that yet, so will see what happens.

I don't really get how multiple users work in terms of pretty much
everything else, like if the Python code is running on the server,
then... well, I just don't know. Maybe I should try to get it running
for multiple discrete users first, and then think about simultaneous
users, or is that a bad way to go about things? Or maybe it will
start to make more sense when I get into building the interface? Any
info/suggestions are very welcome.

Thanks again!

Shel

 

[Shel]

Just want to say thanks again to everyone who responded to this post.
You have all been really helpful.

 

[Martin Gregorie]

I am confused about multiple simultaneous users, which I would like to
be able to accommodate. On the db side, I have a structure to store
data for each user, and know a bit about selectively locking data,
although I have not implemented that yet, so will see what happens.

I realise what I wrote last night wasn't all that clear. Terms:
'transaction' and 'session'.

A web server 'transaction' consists of a request from a user that results
in a page being sent to the user. That's it. It is an isolated action
that does not depend in the web server knowing anything about the user
because all the information it needs to decide which page to send was
supplied when the user sent in the URL of the page. Now if the user
clicks on a link on that page, his browser sends the URL in the link to
the server, which in turn fishes out another page and sends it back to
the user. As far as the server is concerned, there is no connection
whatever between the two requests: either or both URLs could have been
copied from a piece of paper for all it knows or cares. There is no
concept of context or a session involved.

A 'session' involves context. Think of what happens when you login to a
computer. That starts a login session that has context: the computer now
knows who you are and provides context by connecting you to your login
directory and opening some work space which is used to remember which
directory you're in, what commands you issued (so you can look at the
history), etc. The session and its context persists until you log out.

In what you're intending to do, a user will start a session by starting
to use your program and that session will last until the user disconnects
from the session. All the web server knows is that instead of finding a
page on disk some place it passes your user's request to your program and
sends its output, in the form of a web page, back to the user. It does
this each time it receives a request from the user because all the user's
requests contain the same URL - that of your program. The server does
this without knowing there is such a thing as a session or that there is
any context belonging to the user.

The upshot is that your program has to keep track of all the active
sessions and maintain context for each active session. It also needs a
way to recognise and get rid of dead sessions because sessions don't
always end cleanly: the line may go down or the user may forget he was
using your program and turn his PC off. For instance, if the session
context has a timestamp, you might delete it after, say, 20 hours of
inactivity, or when the user logs on again. If the data is sensitive, you
might also force a new logon after 10 minutes of inactivity.

The database is as good a place as any for keeping session and context
data - if its well structured the context may well form a single (large)
row on one table, but you do need a unique key for it. That could even be
the login name provided you're able to include it in every page you send
to the user and can guarantee that the browser will send it back as part
of the next request. A hidden field on the page will do this
automatically.

The basic program cycle will be:

- receive a request
- read the context for the session
- use data in the request to carry out the requested action
- write the updated context back to the database
- create the output page and send it to the user

though of course you need additional dialogue to deal with both valid and
invalid logons and logoffs.

I don't really get how multiple users work in terms of pretty much
everything else, like if the Python code is running on the server,
then... well, I just don't know.

Hopefully the above made it a bit clearer.

Maybe I should try to get it running
for multiple discrete users first, and then think about simultaneous
users, or is that a bad way to go about things? Or maybe it will start
to make more sense when I get into building the interface? Any
info/suggestions are very welcome.

For bare desktop development I would split the program into three parts:

1) the program itself, written to run a single transaction each time its
called. Inputs would be the bits of the users message it needs to act on
and the current session context record.

2) a testing harness that accepts user input from the console, sends
output back to the console and maintains a single session context record
in memory: IOW it runs your program in single user mode.

3)the web server interface which retrieves the session context record,
passes it and the input to your program and, after that has run, saves
the session context record and passes the output to the web server for
delivery to the user.

This way both 2 and 3 can be developed against a really simple 'do almost
nothing' version of 1 while that in turn can be developed and tested on
your desktop using 2 and later be dropped into the web server with 3 as
its interface.

I have an in-house copy of Apache that I'd use to develop your type of
program. Its used for all my website development so that nothing gets
loaded onto my public sites until its been properly checked out here.
You can do the same if you can find and install a really simple web
server that would run on your PC together with a local copy of MySQL - of
course! Given this setup you can use your usual web browser to talk to
the local web server. If you can run all that you won't need 2 because
you can have your simple web server and program running in a console
window on your desktop PC while you hammer it from your web browser.

 

[Jean-Michel Pichavant]

Hello,

I am pretty new to all this. I have some coding experience, and am
currently most comfortable with Python. I also have database design
experience with MS Access, and have just created my first mySQL db.

So right now I have a mySQL db structure and some Python code. My end
goal is to create a browser-based interactive fiction/game thing. My
code is currently just using dummy data rather than pulling in data
from the db, but I think/hope it won't be too big of a deal to
interact with the db through Python (famous last words...).

My main problem right now is how to do the web interface. I don't know
much about web architecture, unfortunately. I think I need a CGI
script?

What I would really like is to find a GUI tool to design the interface
that would have customizable templates or drag-and-drop elements or
something, so I wouldn't have to code much by hand. Something easy, if
such a tool exists.

Also would prefer something that I would be able to install/use
without having much access to the server where the site will be hosted
(on a university server that I don't have control over). I don't fully
understand how a lot of the tools I have been looking at work, but it
seems like they're often things where you'd have to get an
administrator to do stuff on the server (is that right?), which I
would prefer to avoid.

It looks like Django has some sort of standalone implementation, but
I'm not clear on how that would work or how much of a learning curve
there would be for using it.

If anyone has any thoughts or suggestions, I would really appreciate
it.

Hope my questions make sense. I don't really know what I'm doing, so
could be they're a bit silly. I apologize if that's the case, and
please let me know if you need any additional informmation.

Thanks,
Shel

Django is quite popular, they claim to be easy to learn/use. That's for
the web framework.
You could use pyjamas for the web interface, looks like it works well
with django.

Hmm, writing python code only, sounds like a dream come true

Note that I never used one of these, it just happened I had to look for
possible solutions for a web application.


JM

 

[Shel]

This is really great. I wish I could come up with some creative new
ways to say thank you, but... thank you
Shel

 

[Alice Bevan–McGregor]

Howdy!

I'm mildly biased, being the author of the framework, but I can highly
recommend WebCore for rapid prototyping of web applications; it has
templating via numerous template engines, excellent JSON (AJAJ)
support, and support for database back-ends via SQLAlchemy. It also
has session support baked-in via a project called Beaker.
Documentation is fairly complete, and I can be found camping in the
#webcore IRC channel on irc.freenode.net at strange hours.

If you can write a class, you can have a fully operational web
application in a single file of ~8 lines or so. (Or you can create a
complete easy-installable Python package with multiple modules.)

For information, see: http://www.web-core.org/

As an interactive-fiction example:

class RootController(web.core.Controller):
def index(self):
"""This returns a template that uses JavaScript to call execute().
The JavaScript adds the result of execute() to the display."""
session = db.Session().save()
return './templates/main.html', dict(session=session.id)

def execute(self, session, statement):
"""Load our session and pass the input off to our interactive
fiction library of choice. Return the result if all went well."""
session = db.Session.get(session)

try:
result = myiflib.execute(session, statement)

except myiflib.ParseError:
return 'json:', dict(status="failure", message="Error...")

return 'json:', dict(status="success", message=result)

— Alice.

 

[Shel]

Will take a look after stuffing myself with turkey today (am in the
US, where we give thanks by eating everything in sight). Thanks,
Alice.

Wait, did I just say "thanks"? Must go eat pie.