Quantization

[ashu]

Hi Dear
I am working on quantization, for this I need to divide two signed
numbers. The procedure I am using is working fine for most of the
input data but with some values results are not correct. Please
suggest me the remedy. The procedure I am using is as follows:

I. 33/22 = 1.5 =~ 2

II. 33*512\22*512

III. (33 * 23)/512 ( as 512/22 = 23.2 =~ 23)

IV. Binary representation of 33*22 (1011110111) is shifted by nine
bits right (SRA Shift right arithematic) to divide by 512 & if the
last shifted bit i.e. ninth bit is one then result is incremented by
one else leave it as such.

V. In this particular case as ninth bit is 0 thus I did not increment
the result & get the result as one which actually should be two.

It will be a great favor if anybody can help me out.


Thanks & regards
Ashwani

 

[Pascal Peyremorte]

ashu a écrit :

Hi Dear
I am working on quantization, for this I need to divide two signed
numbers. The procedure I am using is working fine for most of the
input data but with some values results are not correct. Please
suggest me the remedy. The procedure I am using is as follows:

I. 33/22 = 1.5 =~ 2

II. 33*512\22*512

III. (33 * 23)/512 ( as 512/22 = 23.2 =~ 23)

IV. Binary representation of 33*22 (1011110111) is shifted by nine
bits right (SRA Shift right arithematic) to divide by 512 & if the
last shifted bit i.e. ninth bit is one then result is incremented by
one else leave it as such.

V. In this particular case as ninth bit is 0 thus I did not increment
the result & get the result as one which actually should be two.

It will be a great favor if anybody can help me out.

Hi,

This is due to the rounding done when you assimilate 512/22 = 23 instead of 23.2

If you look at the binary representation of 33*23, you will found :
1 011110111 (instead of 1 100000000 that is the real value of 1.5 * 512)

Checking only 1 bit make you to round 011110111 to zero instead to 100000000 and report that error in your result.


A better solution to make a right rounding is to add half the divisor to the dividend before doing the division.

You will get (33*23 + 256) / 512 that will give you 2 after the integer division.

Pascal