2D arrays, pointers, and pointer arithmetic: question

[James Kuyper]

Using near or far pointers in any way at all will invoke an undefined
behavior, so I think it's safe to say bad results are likely with or
without attempts to printf() such pointers!

Within the context of the C standard, "undefined behavior" means only
that the C standard doesn't define the behavior. If the compiler's
documentation does define the behavior, you should be OK, even if the C
standard doesn't.

However, it's extremely unlikely that a compiler which supports both
near and far pointers would come along with a C standard library who's
printf() allows you to print both types of pointers with the same "%p"
format specifier. That's the "bad results in practice" that he was
talking about.

 

[Keith Thompson]

Using near or far pointers in any way at all will invoke an undefined
behavior, so I think it's safe to say bad results are likely with or
without attempts to printf() such pointers!

For a C implementation that supports `near`, `far`, `void*`, and
printf's "%p" format, I would expect that casting a `near` or `far`
pointer value to `void*` before passing it to printf would work
"correctly". Of course the standard doesn't guarantee that, but I'd
expect it of any decent C or C-like implementation.

(I've never used `near` or `far` myself. It's plausible that an
implementation old enough to support them would be too old to support
`void*`.)

 

[Les Cargill]

For a C implementation that supports `near`, `far`, `void*`, and
printf's "%p" format, I would expect that casting a `near` or `far`
pointer value to `void*` before passing it to printf would work
"correctly". Of course the standard doesn't guarantee that, but I'd
expect it of any decent C or C-like implementation.

(I've never used `near` or `far` myself. It's plausible that an
implementation old enough to support them would be too old to support
`void*`.)

The C89 flavors of MSC supported both "void*" and "near"/"far". TINY
MODEL FTW!

 

[ais523]

(I've never used `near` or `far` myself. It's plausible that an
implementation old enough to support them would be too old to support
`void*`.)

16-bit versions of Borland C for Windows supported void* just fine
(we're talking well after 1989; they ran under Windows 95 to be able to
compile programs that would run under the still-then-in-use Windows
3.1), but also supported near and far because they existed on that
platform.

The fix used for standards compliance was to have compiler options to
specify whether function pointers were near/far if not specified, and
whether non-function pointers were near/far if not specified. (This
means, among other things, that it was not necessarily possible to
convert a void(*)() to a void* and back again, which makes for a nice
counterexample to give to people who think it is possible.) The system
headers marked every pointer with near or far in order to make them
conform to the library ABI, and implicit casts were added to the
language avoid this causing trouble.

There was actually a third option, huge, which could cope with objects
larger than one segment (by performing extra arithmetic upon segment
overflow), and was otherwise like far. (And was rarely used, because
objects that large were not so useful back then due to limited memory.)
I also remember the system-specific library function farmalloc(), which
was like malloc but which gave you a far pointer; very useful for
storing large amounts of data long-term in a program which otherwise
could do with near pointers for data.

 

[Malcolm McLean]

בת×ריך ×™×•× ×—×ž×™×©×™, 30 ב×וגוסט 2012 02:09:02 UTC+1, מ×ת Les Cargill:

Keith Thompson wrote:

The C89 flavors of MSC supported both "void*" and "near"/"far". TINY
MODEL FTW!

The idea of the "models" was that you could write program without the
near/far non-standard extensions. Under "huge", if I remember rightly,
all pointers were 32 bit and it did some inefficient magic to hide the
segments and give you what looked like flat memory. Under "large",
pointers were 32 bits but you couldn't allocate a block of more than
64 K.
Under "tiny" all code and data had to fit within the same 64K segment,
which led to the most efficient machine code. But you could use far
pointers to allocate data in other segments. So you ended up rewriting
fucntions like "farmemcpy", because the standard library only worked
with the standard pointers.

 

[ralph]

[... about converting int* to void* for "%p" conversion ...]
BTW: I don't know if my misconception is "frequent" enough to warrant an
FAQ change, but it doesn't look like the FAQ covers this. Contra Eric,
5.17 is about something else; 4.17 / 19.40 are closer but not quite
relevant.

5.17 is mostly about representations of null pointers, but
not entirely. If you'll read the entire thing, you'll find two
specific mentions of different pointer formats, plus a link to
further examples.

14.17 and 19.40(d) are about `near' and `far', whose
connection to the matter at hand eludes me.

`near` and `far`, which are of course non-standard, can result in
different pointer types having different sizes. Passing a `near`
or `far` pointer to printf with a "%p" format is likely to give
bad results in practice.

Actually in practice it was never an issue.

Because of how the different models were implemented a different CRT
or "standard C Library" was always employed, thus each encountered a
"printf()" fully able to managed whatever "%p" formatted argument it
received.

-ralph

 

[Keith Thompson]

[...]

`near` and `far`, which are of course non-standard, can result in
different pointer types having different sizes. Passing a `near`
or `far` pointer to printf with a "%p" format is likely to give
bad results in practice.

Actually in practice it was never an issue.

Because of how the different models were implemented a different CRT
or "standard C Library" was always employed, thus each encountered a
"printf()" fully able to managed whatever "%p" formatted argument it
received.

Wasn't it possible to have both near and far pointers, with different
sizes, in the same program?

For example:

char near *nearp = ...;
char far *farp = ...;
printf("nearp = %p, farp = %p\n", nearp, farp);

Maybe that wasn't a common thing to do, but wouldn't it cause problems?

 

[Anders Wegge Keller]

Wasn't it possible to have both near and far pointers, with
different sizes, in the same program?

It is. The 80186 compiler I use at work calls this "Mixed mode".

For example:

char near *nearp = ...;
char far *farp = ...;
printf("nearp = %p, farp = %p\n", nearp, farp);

Maybe that wasn't a common thing to do, but wouldn't it cause problems?

One should think that this would give a funny-looking result for the
near pointer. But I've never had to log pointer values on that
platform, so I cannot say for sure what would happen.

 

[ralph]

[...]

`near` and `far`, which are of course non-standard, can result in
different pointer types having different sizes. Passing a `near`
or `far` pointer to printf with a "%p" format is likely to give
bad results in practice.

Actually in practice it was never an issue.

Because of how the different models were implemented a different CRT
or "standard C Library" was always employed, thus each encountered a
"printf()" fully able to managed whatever "%p" formatted argument it
received.

Wasn't it possible to have both near and far pointers, with different
sizes, in the same program?

For example:

char near *nearp = ...;
char far *farp = ...;
printf("nearp = %p, farp = %p\n", nearp, farp);

Maybe that wasn't a common thing to do, but wouldn't it cause problems?

Not in "common practice" since the keywords "far" and "near" were
themselves implementaton and conditionally defined. ie, "far" was
actually a defined macro "__far"*.

It was essentially difficult to create 'near' and 'far' pointers in
the same translation unit without going out of your way to contrive
something.

char near *nearp = ...;
char far *farp = ...;

I would expect the above to silent or deliver a strong Warning or even
Error, depending on model being compiled.

Of course we are talking about more than a vendor extension to the
language. Microsoft besides simply providing two different sized
pointer declarations also did thunking and other redefinitions behind
the scenes. (Both the preprocessor and linker were very busy little
beavers. <g>) For example, 'near' pointers magically became 'far'
pointers when calling something in a 'WinAPI' shared library. Well not
too much magic. They basically just prefixed the current defaut
segment to the near address. Hard to say what else might have been
going on. Can't remember all of it.

Do remember that it 'just worked'. <g>

-ralph
[__far* Originally it was "_far", one under-score. Later it become
"__far" to be ANSI compliant for vendor implementation defines. The
final definition was often buried in a stack of #defines. <g>]

 

[Richard Damon]

בת×ריך ×™×•× ×—×ž×™×©×™, 30 ב×וגוסט 2012 02:09:02 UTC+1, מ×ת Les Cargill:
The idea of the "models" was that you could write program without the
near/far non-standard extensions. Under "huge", if I remember rightly,
all pointers were 32 bit and it did some inefficient magic to hide the
segments and give you what looked like flat memory. Under "large",
pointers were 32 bits but you couldn't allocate a block of more than
64 K.
Under "tiny" all code and data had to fit within the same 64K segment,
which led to the most efficient machine code. But you could use far
pointers to allocate data in other segments. So you ended up rewriting
fucntions like "farmemcpy", because the standard library only worked
with the standard pointers.

The models came about due to the memory structure of the 16 bit x86
architecture. Addresses were in general greater than 16 bits in length,
while address registers were only 16 bits long, and were combined with a
"segment register" to convert the value to a full address. (depending on
what mode the processor was in would change how the conversion was done).

Near pointers only stored the value of the address register, and the
segment register was assumed by the type of pointer (the Data Segment
Register for "data" pointers, and the Program Segment Register for
function pointers). In addition, data pointers had a huge type, which
was like a far pointer, but could point to an object that might be
bigger than a single segment, and the compiler would need to do
additional work on address arithmetic to handle this. It mostly was used
in "Real" mode, where the Segment register was just added to the address
register after shifting up the Segment register 4 bits (giving a 20 bit
final memory address).

The program model determined what were the default sizes for each type
of pointer.

Model Data Code
Tiny near near
Small near near
Medium near far
Compact far near
Large far far
Huge huge far

The difference between Tiny and Small was that in Tiny, the code and
data were in the same segment, while in Small they were in distinct
segments.

It was by far more common to have near/far pointers in code that had a
default near size of pointers, to handle a limited number of
objects/functions that would be placed outside the default near block to
make room for them, sometimes to access things outside the current program.

Note also that the 32 bit x86 family of processors still have these
memory models (and 48 bit "far" pointers), they are just mostly ignored
and most programs are just done in the Tiny or Small model, after all
who should need more than 4GB of address space The resurgence of
memory models for programs was headed off (for now at least) with the
introduction of 64 bit processors.

 

[88888 Dihedral]

The models came about due to the memory structure of the 16 bit x86

architecture. Addresses were in general greater than 16 bits in length,

while address registers were only 16 bits long, and were combined with a

"segment register" to convert the value to a full address. (depending on

what mode the processor was in would change how the conversion was done).



Near pointers only stored the value of the address register, and the

segment register was assumed by the type of pointer (the Data Segment

Register for "data" pointers, and the Program Segment Register for

function pointers). In addition, data pointers had a huge type, which

was like a far pointer, but could point to an object that might be

bigger than a single segment, and the compiler would need to do

additional work on address arithmetic to handle this. It mostly was used

in "Real" mode, where the Segment register was just added to the address

register after shifting up the Segment register 4 bits (giving a 20 bit

final memory address).



The program model determined what were the default sizes for each type

of pointer.



Model Data Code

Tiny near near

Small near near

Medium near far

Compact far near

Large far far

Huge huge far



The difference between Tiny and Small was that in Tiny, the code and

data were in the same segment, while in Small they were in distinct

segments.



It was by far more common to have near/far pointers in code that had a

default near size of pointers, to handle a limited number of

objects/functions that would be placed outside the default near block to

make room for them, sometimes to access things outside the current program.



Note also that the 32 bit x86 family of processors still have these

memory models (and 48 bit "far" pointers), they are just mostly ignored

and most programs are just done in the Tiny or Small model, after all

who should need more than 4GB of address space The resurgence of

memory models for programs was headed off (for now at least) with the

introduction of 64 bit processors.

These are old tricks. Now there are 4 to 8 core cpus in the tablet computers,
or note book computers running Androids( LINUX+JAVA by a new name for marketing).

How to support the multi-core part in C is still not addressed very well?

 

[Richard Damon]

How to support the multi-core part in C is still not addressed very well?

Until the latest standard came out, "ISO Standard C" had no concept of
"threads", and without threads, the issues of a multi-core CPU are
irreverent. Any program that wanted to use threads had to rely on some
other standard of some sort to implement the threads and that standard
needed to define how to handle the various issues with multiple cores.

There is no inherent problem with this, most real programs actually are
going to depend on some standard beyond "ISO Standard C", if only the
documentation of the compiler, many times on something like POSIX or the
like.

With the new standardization of some basic thread supporting primitives
in ISO C, the capability to write more stuff in strictly conforming ISO
Standard C is available (once support becomes common), so perhaps some
programs that used to have a requirement on say POSIX, can now just
require C11 compliance, but most are still going to be dependent on
things beyond the ISO C standard.

I would suspect that in reality, most code for a program should be
fairly agnostic to the multi-core nature of the machine, as it should be
dealing with data that it currently "owns" and the interfaces that pass
data from one thread to another should deal with all the needed memory
barriers to make sure this "just works". It is only the sections that
need to deal with actually sending information between threads that
needs to worry about these aspects, and in most cases the threading
library will provide a "good enough" interface (if a bit heavy). It is
only the case where trying to improve efficiency that the applications
programmer is going to need to deal with these details, and in most
cases when dealing with improving efficiency, there is a tendency to
move into implementation defined/specific characteristics anyway (how
expensive things are tends to be a function of the implementation).

 

[Tim Rentsch]

On Mon, 27 Aug 2012 13:35:35 -0700 (PDT), (e-mail address removed) wrote: [snip]

The i term added to a is being added to the base address of the array
and itself, i, representing the size of a row. j then adds to that
address giving the column position.

When a pointer and an integer are added, the value of the integer is
scaled by the size of the object pointed to, or if you prefer, the
size of the object type pointed to.

Another way to look at it is that the value is *not* "scaled" by the
size of the object pointed to; rather, the addition yields a pointer
that points N *objects* past the object that the original pointer
points to. (There has to be an array for this to make sense, possibly
the 1-element array that's equivalent to a single object.)

The standard's description doesn't talk about scaling (N1570 6.5.6p8):
[snip quoted paragraph]

But, talking about the [] array indexing operator, the Standard
does say in a footnote

Then i is adjusted according to the type of x, which
conceptually entails multiplying i by the size of the
object to which the pointer points

I'm not arguing for or against either explanation; I think both
have some support (as an explanation) in the Standard.