char * containing all kind of types

D

David RF

His folks, is this safe code?

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(void)
{
char c = 'A'; int i = 1; float f = -1.01; double d =
1000000.99999;
char *pc; int *pi; float *pf; double *pd;
char *p, *data;

p = data = malloc(sizeof(char) + sizeof(int) + sizeof(float) +
sizeof(double));
if (data == NULL) {
fprintf(stderr, "out of mem\n");
exit(1);
}

memcpy(data, &c, sizeof(c));
data += sizeof(c);
memcpy(data, &i, sizeof(i));
data += sizeof(i);
memcpy(data, &f, sizeof(f));
data += sizeof(f);
memcpy(data, &d, sizeof(d));
data = p;

pc = (char *)data;
printf("%c\n", *pc);
data += sizeof(c);

pi = (int *)data;
printf("%d\n", *pi);
data += sizeof(i);

pf = (float *)data;
printf("%f\n", *pf);
data += sizeof(f);

pd = (double *)data;
*pd = 15.78995455;
printf("%lf\n", *pd);

data = p;
free(data);
return 0;
}
 
I

Ike Naar

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(void)
{
char c = 'A'; int i = 1; float f = -1.01; double d =
1000000.99999;
char *pc; int *pi; float *pf; double *pd;
char *p, *data;

p = data = malloc(sizeof(char) + sizeof(int) + sizeof(float) +
sizeof(double));
if (data == NULL) {
fprintf(stderr, "out of mem\n");
exit(1);

small nit: ``exit(EXIT_FAILURE);'' is more portable
}

memcpy(data, &c, sizeof(c));
data += sizeof(c);
memcpy(data, &i, sizeof(i));
data += sizeof(i);
memcpy(data, &f, sizeof(f));
data += sizeof(f);
memcpy(data, &d, sizeof(d));
data = p;

pc = (char *)data;
printf("%c\n", *pc);
data += sizeof(c);

pi = (int *)data;

Not safe; data may be improperly aligned for an int* .
printf("%d\n", *pi);
data += sizeof(i);

pf = (float *)data;

Not safe; data may be improperly aligned for a float* .
printf("%f\n", *pf);
data += sizeof(f);

pd = (double *)data;

Not safe; data may be improperly aligned for a double* .
*pd = 15.78995455;
printf("%lf\n", *pd);

The format for printing a double is just "%f", not "%lf".
data = p;
free(data);

Why not ``free(p);'' ?
 
N

Nick Keighley

On 25 mayo, 11:21, Ian Collins <[email protected]> wrote:

CONTEXT:

****
[data is a char*]
memcpy(data, &c, sizeof(c));
data += sizeof(c);
memcpy(data, &i, sizeof(i));
data += sizeof(i); [...]
pc = (char *)data;
printf("%c\n", *pc);
data += sizeof(c);

pi = (int *)data;
***

[try unsigned char?]
Yes, stupid question ...

is there a portable way?

to do what? If you want to reuse memory try a union- it will be
aligned correctly for the most-fussy-type. If you want to store
different types in order use a struct- which will add padding to solve
the alignment problem. malloc()d memory is guranteed to be correctly
aligned for any type, so a linked list of malloc()d blocks might work.
Some implementaions use the union trick to get the memory correctly
aligned for the malloc()d memory.

A char* can't hold all types. It can't (portably) hold a ptr-to-
function.
 
M

Mark Bluemel

Yes, stupid question ...

is there a portable way?

What are you actually trying to achieve?

You can't expect to access unaligned data as anything other than
bytes, so you need to align the data either as it's stored - perhaps
by using a struct - or when you access it, say by reversing the
memcpy() calls you used to store it.
 
D

David RF

to do what?

Ok, sorry, I'm trying to pack n (unknow at compile time) values
(stored in a binary file) containing different types, into a single
container, actually I use a *void[], how to avoid that?

Excuse my poor english.
 
P

Paul Heininger

to do what?

Ok, sorry, I'm trying to pack n (unknow at compile time) values
(stored in a binary file) containing different types, into a single
container, actually I use a *void[], how to avoid that?
<snip>

Why are you not using a "union"?
 
D

David RF

Why are you not using a "union"?

Thanks Paul,
Yes, I know union is the better way, but binary file can be very very
big, It's a pain to spend 8 bytes for store a 1 byte char or 4 bytes
for int or float
 
B

Ben Bacarisse

David RF said:
to do what?

Ok, sorry, I'm trying to pack n (unknow at compile time) values
(stored in a binary file) containing different types, into a single
container, actually I use a *void[], how to avoid that?

I don't know what you mean by *void[] but you can unpack the data in the
same way that you packed it -- using memcpy.

If you need the code to be portable, you might have switch from using
memcpy to packing and unpacking the bytes of the objects yourself, since
different machines have different ideas about how an object's bytes are
ordered. Of course, this can raise other portability issues (i.e. some
machines don't have 8-but bytes) so it is a good idea to know something
about the range of machines the code will run on.
 
D

David RF

I don't know what you mean by *void[] but you can unpack the data in the
same way that you packed it -- using memcpy.

In this way? that's portable?

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(void)
{
char c = 'A'; int i = 1; float f = -1.01; double d = 1000000.99999;
char *p, *data;

p = data = malloc(sizeof(c) + sizeof(i) + sizeof(f) + sizeof(d));
if (data == NULL) {
fprintf(stderr, "out of mem\n");
exit(1);
}

memcpy(data, &c, sizeof(c));
data += sizeof(c);
memcpy(data, &i, sizeof(i));
data += sizeof(i);
memcpy(data, &f, sizeof(f));
data += sizeof(f);
memcpy(data, &d, sizeof(d));
data = p;

memcpy(&c, data, sizeof(c));
printf("%c\n", c);
data += sizeof(c);
memcpy(&i, data, sizeof(i));
printf("%d\n", i);
data += sizeof(i);
memcpy(&f, data, sizeof(f));
printf("%f\n", f);
data += sizeof(f);
memcpy(&d, data, sizeof(d));
printf("%f\n", d);

free(p);
return 0;
}

Thanks
 
B

Ben Bacarisse

David RF said:
I don't know what you mean by *void[] but you can unpack the data in the
same way that you packed it -- using memcpy.

In this way? that's portable?
char c = 'A'; int i = 1; float f = -1.01; double d = 1000000.99999;
char *p, *data;
memcpy(data, &c, sizeof(c));
data += sizeof(c);
memcpy(data, &i, sizeof(i));
data += sizeof(i);
memcpy(data, &f, sizeof(f));
data += sizeof(f);
memcpy(data, &d, sizeof(d));
data = p;

memcpy(&c, data, sizeof(c));
printf("%c\n", c);
data += sizeof(c);
memcpy(&i, data, sizeof(i));
printf("%d\n", i);
data += sizeof(i);
memcpy(&f, data, sizeof(f));
printf("%f\n", f);
data += sizeof(f);
memcpy(&d, data, sizeof(d));
printf("%f\n", d);
<snip>

Yes, that's portable but, as you know, largely pointless -- you could
just keep the data in the variables.

What no one here but you knows, is why you are doing this. If the
unpacking may happen on a different machine to the packing (say because
it is being sent over a network) then the code is no longer fully
portable. I'd say more, but it seems silly to guess what it is you are
doing.
 
D

David RF

Yes, that's portable but, as you know, largely pointless -- you could
just keep the data in the variables.

What no one here but you knows, is why you are doing this.  If the
unpacking may happen on a different machine to the packing (say because
it is being sent over a network) then the code is no longer fully
portable.  I'd say more, but it seems silly to guess what it is you are
doing.

I'm trying to load a large set of data from a binary file into a data
container,
as said, binary can be very very big and I can't build a struct
because I don't
know how many fields (and his types) are stored in the binary at
compile time,
I know I can use a union but I don't wan't to spend more than
neccesary.
I don't believe that some one can understand this spanglish text :)
Thanks again!!
 
B

Ben Bacarisse

David RF said:
I'm trying to load a large set of data from a binary file into a data
container, as said, binary can be very very big and I can't build a
struct because I don't know how many fields (and his types) are stored
in the binary at compile time, I know I can use a union but I don't
wan't to spend more than neccesary.

I don't see how using a union here will help. Someone suggested using a
union because you showed both packing and unpacking code. If the data
is already in a file, you just have to extract it and a union won't
help.

The fact that the data is in a file does not tell me much. If you think
that issues of byte ordering won't matter to you, then go ahead and use
memcpy. If you are not sure, then you need to say more. Specifically
we would need to know how this file comes about (do you write it? is
the format fixed say by some specification?) and whether your code needs
to be portable and, if so, across which machines (or at least what sort
of machines).
I don't believe that some one can
understand this spanglish text :)

You are perfectly understandable, but you are not saying enough to give
really helpful answers.
 
D

David RF

You are perfectly understandable, but you are not saying enough to give
really helpful answers.


You're very kind
Despite binary file, here in ascii, I have a file named data.csv and
another named data.def

data.csv contains data
data.def contains definition of each field

data.csv
H,5,25,79.911
E,4,24,27.553
J,8,11,51.952
K,8,22,14.606
D,3,4,80.156
C,1,3,99.218
L,10,19,29.637
H,6,31,29.771
L,10,12,89.283
L,10,29,6.949

data.def
0113
# 0 char | 1 int | 2 float | 3 double

both files are unknown at compile time

I wan to load all data without spend 8 bytes for each field in the
struct

Ok, it's very hard to me to explain, let speak to C:
Note: This code uses flexible array, must be compiled with c99 flag


#include <stdio.h>
#include <string.h>
#include <stdlib.h>

enum {type_chr, type_int, type_flt, type_dbl};

#define FILE_LINE_SIZE 100

typedef struct TTable {
struct TCol *col_first;
struct TRow *row_first;
struct TCol *col_last;
struct TRow *row_last;
int cols;
int rows;
} Table;

typedef struct TCol {
struct TTable *table;
struct TCol *prev;
struct TCol *next;
int id;
int type;
} Col;

typedef union TData {
char as_chr;
int as_int;
float as_flt;
double as_dbl;
} Data;

typedef struct TRow {
struct TTable *table;
struct TRow *prev;
struct TRow *next;
Data data[]; /* Array flexible */
} Row;

static Table *table_create()
{
Table *new;

new = malloc(sizeof(Table));
if (new == NULL) {
fprintf(stderr, "Nos hemos quedado sin memoria.");
exit(EXIT_FAILURE);
}
new->col_first = NULL;
new->row_first = NULL;
new->col_last = NULL;
new->row_last = NULL;
new->cols = 0;
new->rows = 0;
return new;
}

static Col *table_add_col(Table *table, int type)
{
Col *new;

new = malloc(sizeof(Col));
if (new == NULL) {
fprintf(stderr, "Nos hemos quedado sin memoria.");
exit(EXIT_FAILURE);
}
new->table = table;
new->id = table->cols++;
new->type = type;
if (table->col_last) {
table->col_last->next = new;
new->prev = table->col_last;
} else {
table->col_first = new;
new->prev = NULL;
}
new->next = NULL;
table->col_last = new;
return new;
}

static Row *table_add_row(Table *table)
{
Row *new;

new = malloc(sizeof(Row) + (sizeof(Data) * table->cols));
if (new == NULL) {
fprintf(stderr, "Nos hemos quedado sin memoria.");
exit(EXIT_FAILURE);
}
new->table = table;
if (table->row_last) {
table->row_last->next = new;
new->prev = table->row_last;
} else {
table->row_first = new;
new->prev = NULL;
}
new->next = NULL;
table->row_last = new;
table->rows++;
return new;
}

static Row *table_delete_row(Row *row)
{
Row *res = NULL;

if (row) {
if (row->prev) {
row->prev->next = row->next;
} else {
row->table->row_first = row->next;
}
if (row->next) {
row->next->prev = row->prev;
} else {
row->table->row_last = row->prev;
}
res = row->next;
row->table->rows--;
free(row);
}
return res;
}

static void table_clear(Table *table)
{
while ((table->row_first = table_delete_row(table->row_first)));
}

static void table_free(Table *table)
{
Col *col;

table_clear(table);
while (table->col_first) {
col = table->col_first->next;
free(table->col_first);
table->col_first = col;
}
free(table);
}

static void table_raise_error(Table *table, const char *err)
{
table_free(table);
fprintf(stderr, "%s", err);
exit(EXIT_FAILURE);
}

static void table_print(Table *table)
{
Row *row;
Col *col;

for (row = table->row_first; row; row = row->next) {
printf("-\n");
for (col = table->col_first; col; col = col->next) {
switch (col->type) {
case type_chr:
printf("\tCampo %d = %c\n", col->id, row->data[col-
id].as_chr);
break;
case type_int:
printf("\tCampo %d = %d\n", col->id, row->data[col-
id].as_int);
break;
case type_flt:
printf("\tCampo %d = %f\n", col->id, row->data[col-
id].as_flt);
break;
case type_dbl:
printf("\tCampo %d = %f\n", col->id, row->data[col-
id].as_dbl);
break;
}
}
}
}

int main(void)
{
Table *table;
Col *col;
Row *row;
FILE *file;
char c, s[FILE_LINE_SIZE], *p;

table = table_create();
file = fopen("data.def", "r");
if (file == NULL) {
table_raise_error(table, "No se puede abrir el archivo data.def
en modo lectura.\n");
}
/* Por cada tipo declarado en data.def ... */
/* For each type in data.def */
while ((c = fgetc(file))) {
if ((c < '0') || (c > '3')) break;
/* ... creo una columna */
/* ... create a col */
table_add_col(table, c - '0');
};
fclose(file);
if (table->cols == 0) {
table_raise_error(table, "Sin campos no puedo hacer mucho!!!\n");
}
/* Leo data.csv cargando la estructura */
/* Load data.csv */
file = fopen("data.csv", "r");
if (file == NULL) {
table_raise_error(table, "No se puede abrir el archivo data.csv
en modo lectura.\n");
}
while ((p = fgets(s, FILE_LINE_SIZE, file)) != NULL) {
row = table_add_row(table);
col = table->col_first;
while (col) {
switch (col->type) {
case type_chr: sscanf(p, "%c", &row->data[col->id].as_chr);
break;
case type_int: sscanf(p, "%d", &row->data[col->id].as_int);
break;
case type_flt: sscanf(p, "%f", &row->data[col->id].as_flt);
break;
case type_dbl: sscanf(p, "%lf", &row->data[col->id].as_dbl);
break;
}
if ((col = col->next)) {
while (*++p != ',');
p++;
}
}
}
fclose(file);
if (table->rows == 0) {
table_raise_error(table, "Sin datos no puedo hacer mucho!!!\n");
}
table_print(table);
table_free(table);
return 0;
}
 
N

Nick Keighley

I'm trying to load a large set of data from a binary file into a data
container,

where did the binary data come from? This matters as byte order and
data size might be affected. And even worse things with floating point
are possible (though with these likely it'll be some IEEE format).
as said, binary can be very very big and I can't build a struct
because I don't know how many fields (and his types) are stored
in the binary at compile time, I know I can use a union but I don't
wan't to spend more than neccesary.

I think you're stuffed [English: in a bad position] then. If you read
a 4 byte integer (we'll assume a byte is 8 bits for this discussion)
it *must* be correctly aligned (and endianed) if you want to read it
out of your container as a C int type (I'm including long and short in
"int type"). "Correctly aligned" can mean "on an even byte" or "on an
address divisible by 4". or worse! Some platforms just get slower on
unaligned access (sometimes much slower) some crash.

1. abandon portability and use a platform that tolerates unaligned
access

2. use the union trick and waste a bit of memory

3. use a separate store for each type and store an index

4. store it unaligned and align it on retrieval

all of these are going to have to use something you haven't discussed
yet. Where's y' meta data then matie? In other words how do you know
what data type is stored in the container?
I don't believe that some one can understand this spanglish text :)

you believe that no one can understand this spanglish text?

/* option 3 might look like this */
typedef union {...} Any;

void process_any_item (index i)
{
switch (meta.type)
{
case INT:
{
int data;
data = int_store [meta.index];
process_int_item (data);
}
break

....
}


How are you accessing the container?
 
E

Eric Sosman

You're very kind
Despite binary file, here in ascii, I have a file named data.csv and
another named data.def

data.csv contains data
data.def contains definition of each field

data.csv
H,5,25,79.911
E,4,24,27.553
J,8,11,51.952
K,8,22,14.606
D,3,4,80.156
C,1,3,99.218
L,10,19,29.637
H,6,31,29.771
L,10,12,89.283
L,10,29,6.949

data.def
0113
# 0 char | 1 int | 2 float | 3 double

both files are unknown at compile time

I wan to load all data without spend 8 bytes for each field in the
struct

Re-stating, in case I've misunderstood you: If wishes were
horses, you'd like to read "data.def" and generate a struct type
from it, then read "data.csv" to populate instances of that struct.
Some languages can do that sort of thing, but C cannot: C is a
language whose types are fixed at compile time and immutable
thereafter. You can't create new types at run time.

There are, however, a few things you can do. None is as
convenient as the unachievable build-on-the-fly ideal, and each
has its own drawbacks. Here are a few possibilities:

- Use a "helper" program that reads "data.def" and generates
source code for the real program, then compile the real
program and run it against "data.csv". If the parts of the
real program that deal directly with the data are or can be
fairly well isolated, the "helper" may only need to generate
a smallish part of the total real program, perhaps just a
header file and a description of the struct layout.

- Read "data.csv" and convert its fields as directed by "data.def",
then pack the bytes of each converted row into a sort of of blob,
just jamming them together in memory. In your example, each
blob might occupy 1+4+4+8=17 bytes. To get at the data, you'll
need to pluck it out of its blob and put it in an array of
unions or some such, but you only need enough union arrays to
handle the blob(s) being worked on at the moment, not the
whole set.

- Store the data column-wise instead of row-wise. That is, instead
of somehow building a struct and using array[n].c, array[n].i,
array[n].f, and array[n].d to hold one row's data, you'd use
carray[n], iarray[n], farray[n], and darray[n]. The pointers to
these arrays would probably be in unions, but now you need only
one union per column instead of one per field.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,954
Messages
2,570,116
Members
46,704
Latest member
BernadineF

Latest Threads

Top