Help with strtok

[manochavishal]

Hi
I am writing a Program
in which i get input as

#C1012,S,A#C1013,S,U

I want to get C1012,S,A using strtok and then pass this to function
CreateVideo
which will further strtok this (C1012,S,A) and store the required
values.

Now here is the piece of that code:

#define DELIM2 #
char * field;
char fieldcopy[20];

/*Here i have input as #C1012,S,A#C1013,S,U*/
field = strtok(NULL,DELIM2);
while(field != NULL)
{
strcpy(fieldcopy,field);
CreateCopies(copy,fieldcopy,NoCopies);
field = strtok(NULL,DELIM2);
printf("Field in CreateVideo is %s\n",field);
}

Now if I call CreateCopies the strtok doesn't tokenize till the end.
But if i comment the CreateCopies call it does tokenize till the end.
In the first case the second time i call strtok field gets a value of
NULL but works fine if i dont call CreateObjects.

Why this behaviour???

 

[Richard G. Riley]

Hi
I am writing a Program
in which i get input as

#C1012,S,A#C1013,S,U

I want to get C1012,S,A using strtok and then pass this to function
CreateVideo
which will further strtok this (C1012,S,A) and store the required
values.

Now here is the piece of that code:

#define DELIM2 #

you should change this DELIM2 to use a string.

char * field;
char fieldcopy[20];

/*Here i have input as #C1012,S,A#C1013,S,U*/
field = strtok(NULL,DELIM2);

Its not clear if you have called strtok with your field as first
argument before.

while(field != NULL)
{
strcpy(fieldcopy,field);
CreateCopies(copy,fieldcopy,NoCopies);
field = strtok(NULL,DELIM2);
printf("Field in CreateVideo is %s\n",field);
}

Now if I call CreateCopies the strtok doesn't tokenize till the end.
But if i comment the CreateCopies call it does tokenize till the end.
In the first case the second time i call strtok field gets a value of
NULL but works fine if i dont call CreateObjects.

Why this behaviour???

You should post all the code, theres not enough info there :
specifically the set up of "field" and the initial call to strtok which
should have "field" as the first argument. Since from the look of it,
and assuming nothing nasty, Createfields doesnt change "field" then
the *current* problem is nothing to do with Createfields : could be some bad
pointers because strtok wasnt initialised properly.

 

[Richard Bos]

field = strtok(NULL,DELIM2);
while(field != NULL)
{
strcpy(fieldcopy,field);
CreateCopies(copy,fieldcopy,NoCopies);
field = strtok(NULL,DELIM2);
printf("Field in CreateVideo is %s\n",field);
}

Now if I call CreateCopies the strtok doesn't tokenize till the end.
But if i comment the CreateCopies call it does tokenize till the end.

Does CreateCopies perhaps also use strtok()? You can't nest uses of
strtok(), because it keeps only a single, static state.

Richard

 

[Charles Richmond]

Does CreateCopies perhaps also use strtok()? You can't nest uses of
strtok(), because it keeps only a single, static state.

Well, you can hardly nest invocations of strtok(), but it is *not*
difficult to "roll your own" that *will* support nested invocations.

 

[Richard Bos]

Well, you can hardly nest invocations of strtok(), but it is *not*
difficult to "roll your own" that *will* support nested invocations.

That is true. While you're at it, allow it to split "name,address,,city"
into four rather than three fields, as well.

--

That, btw, is not a .sig-sep.

Richard

 

[CBFalconer]

Well, you can hardly nest invocations of strtok(), but it is *not*
difficult to "roll your own" that *will* support nested invocations.

Here's a version I posted a while ago. Public Domain.

/* ------- file toksplit.h ----------*/
#ifndef H_toksplit_h
# define H_toksplit_h

# ifdef __cplusplus
extern "C" {
# endif

#include <stddef.h>

/* copy over the next token from an input string, after
skipping leading blanks (or other whitespace?). The
token is terminated by the first appearance of tokchar,
or by the end of the source string.

The caller must supply sufficient space in token to
receive any token, Otherwise tokens will be truncated.

Returns: a pointer past the terminating tokchar.

This will happily return an infinity of empty tokens if
called with src pointing to the end of a string. Tokens
will never include a copy of tokchar.

released to Public Domain, by C.B. Falconer.
Published 2006-02-20. Attribution appreciated.
*/

const char *toksplit(const char *src, /* Source of tokens */
char tokchar, /* token delimiting char */
char *token, /* receiver of parsed token */
size_t lgh); /* length token can receive */
/* not including final '\0' */

# ifdef __cplusplus
}
# endif
#endif
/* ------- end file toksplit.h ----------*/

/* ------- file toksplit.c ----------*/
#include "toksplit.h"

/* copy over the next token from an input string, after
skipping leading blanks (or other whitespace?). The
token is terminated by the first appearance of tokchar,
or by the end of the source string.

The caller must supply sufficient space in token to
receive any token, Otherwise tokens will be truncated.

Returns: a pointer past the terminating tokchar.

This will happily return an infinity of empty tokens if
called with src pointing to the end of a string. Tokens
will never include a copy of tokchar.

A better name would be "strtkn", except that is reserved
for the system namespace. Change to that at your risk.

released to Public Domain, by C.B. Falconer.
Published 2006-02-20. Attribution appreciated.
*/

const char *toksplit(const char *src, /* Source of tokens */
char tokchar, /* token delimiting char */
char *token, /* receiver of parsed token */
size_t lgh) /* length token can receive */
/* not including final '\0' */
{
if (src) {
while (' ' == *src) *src++;

while (*src && (tokchar != *src)) {
if (lgh) {
*token++ = *src;
--lgh;
}
src++;
}
if (*src && (tokchar == *src)) src++;
}
*token = '\0';
return src;
} /* toksplit */

#ifdef TESTING
#include <stdio.h>

#define ABRsize 6 /* length of acceptable token abbreviations */

int main(void)
{
char teststring[] = "This is a test, ,, abbrev, more";

const char *t, *s = teststring;
int i;
char token[ABRsize + 1];

puts(teststring);
t = s;
for (i = 0; i < 4; i++) {
t = toksplit(t, ',', token, ABRsize);
putchar(i + '1'); putchar(':');
puts(token);
}

puts("\nHow to detect 'no more tokens'");
t = s; i = 0;
while (*t) {
t = toksplit(t, ',', token, 3);
putchar(i + '1'); putchar(':');
puts(token);
i++;
}

puts("\nUsing blanks as token delimiters");
t = s; i = 0;
while (*t) {
t = toksplit(t, ' ', token, ABRsize);
putchar(i + '1'); putchar(':');
puts(token);
i++;
}
return 0;
} /* main */

#endif
/* ------- end file toksplit.c ----------*/

--
"If you want to post a followup via groups.google.com, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson
More details at: <http://cfaj.freeshell.org/google/>
Also see <http://www.safalra.com/special/googlegroupsreply/>

 

[John Tsiombikas (Nuclear / Mindlapse)]

That is true. While you're at it, allow it to split "name,address,,city"
into four rather than three fields, as well.

Should it also split "name address city" to 7 fields as well if ' '
is part of its seperator string?

Obviously, which one is preferable is highly dependant on what you want
to do with it.

 

[Jordan Abel]

Should it also split "name address city" to 7 fields as well if ' '
is part of its seperator string?

Obviously, which one is preferable is highly dependant on what you want
to do with it.

So there should be two functions. or maybe a flag to be passed to the
single function.

 

[CBFalconer]

So there should be two functions. or maybe a flag to be passed to the
single function.

If you simply say that leading blanks in a token are absorbed, and
that the token char is a separator, the conflicts disappear. See
my toksplit routine elsethread. Which, BTW, is re-entrant.

--
"If you want to post a followup via groups.google.com, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson
More details at: <http://cfaj.freeshell.org/google/>
Also see <http://www.safalra.com/special/googlegroupsreply/>