Discussion regarding hot/ cold splitting of structures.

[Rakesh]

Hi,
I was 'googling' to look out for some ways of optimizing the code
and came across this term - 'hot / cold splitting'.
In short, the discussion is about splitting heavily accessed ( hot )
portions of data structure from rarely accessed cold portions.
I haven't used this one myself anytime before, but am interested in
learning more about this.

Can you please share your experience here, so that I can understand
better and this could be an orientation to write performance-critical
code.

- Rakesh

 

[Michiel Salters]

Hi,
I was 'googling' to look out for some ways of optimizing the code
and came across this term - 'hot / cold splitting'.
In short, the discussion is about splitting heavily accessed ( hot )
portions of data structure from rarely accessed cold portions.
I haven't used this one myself anytime before, but am interested in
learning more about this.

Can you please share your experience here, so that I can understand
better and this could be an orientation to write performance-critical
code.

Assume you have a class C, which is 400 bytes. 300 of those 400 bytes
are needed only to contruct an exception object if C::foo() throws.
Is it smart to keep these 300 bytes directly as members? An alternative
would be to put those 300 bytes in an C::C_exceptiondata class, and put
a C_exceptiondata * in C. The total memory usage would increase
( by sizeof(C_exceptiondata*) ), but the C_exceptiondata objects may
very well end up all swapped out, and certainly would not disrupt the
CPU cache. The cache would only load the remaining 104 bytes.

The 300 bytes that are rarely used are called "cold".

Regards,
Michiel Salters

 

[Ryan]

Hi,
I was 'googling' to look out for some ways of optimizing the code
and came across this term - 'hot / cold splitting'.
In short, the discussion is about splitting heavily accessed ( hot )
portions of data structure from rarely accessed cold portions.
I haven't used this one myself anytime before, but am interested in
learning more about this.

Can you please share your experience here, so that I can understand
better and this could be an orientation to write performance-critical
code.

- Rakesh

Hi Rakesh,

I've never used it either, but it sounds like by keeping heavily
accessed data members near each other physically in memory a system's
caching can yield better performance? I wonder if this assists the
compiler's optimizations at all as well. Maybe someone with more C++
compiler experience can comment.

Ryan

 

[Rakesh]

Thanks Michael for the reply.

So I guess the splitting ought to be performed statically by the
programmer to get the best out of it.

I guess in an ideal world, there could be a compiler, wherein i comment
a field as hot / cold near its declaration and the compiler would be
smart enough to generate code accordingly.

- Rakesh