Someone in the lab recently came to me asking for help in diagnosing a strange problem. The time output from their program was drifting inexplicably — that is, they were adding a certain increment every iteration, but the time was changing by some value slightly different from that increment. It turns out that this was because they were using single-precision floating-point (a.k.a “float”) variables. The problem did not show up right away. In fact, it would not have been an issue were they not adding an increment so very many times.
To diagnose this problem, (because I haven’t dealt with this issue in a long while), I wrote this little piece of code:
#include <stdio.h>
int main(void){
float float_v, fdt;
double double_v, cross_v, ddt;
int i, found;
float_v = 0;
double_v = 0;
cross_v = 0;
found = 0;
ddt = 1e-5;
fdt = 1e-5;
for(i=0; i< 10000; i++){
double_v += ddt;
float_v += fdt;
cross_v += fdt;
if(i%100 == 0){
printf("Float: %5.10f ", float_v);
printf("Doubl: %5.10f ", double_v);
printf("Cross: %5.10f ", cross_v);
printf("Diff : %5.10f\n", double_v - float_v);
}
/* use this to find how many iterations before it screws up
if(double_v - float_v > 1e-8 && !found){
printf("Found discrepancy of %5.10f at step %d!\n", \
double_v - float_v, i);
found = 1;
}
*/
}
return 0;
}
(You can download the code here: add_test.c)
This code tests three combinations:
- Float counter with float increment
- Double counter with double increment
- Double counter with float increment
The little commented section (starts with /* and ends with */) will (if uncommented) stop the program when the difference between the all-double value and the all-float value exceeds 10-8. On Mac OS X on a 64-bit Intel architecture, this happens after 912 steps:
Found discrepancy of 0.0000000101 at step 912!
Here’s the output from 900 iterations (printed every 100):
Float: 0.0000100000 Doubl: 0.0000100000 Cross: 0.0000100000 Diff : 0.0000000000 Float: 0.0010100005 Doubl: 0.0010100000 Cross: 0.0010100000 Diff : -0.0000000005 Float: 0.0020099971 Doubl: 0.0020100000 Cross: 0.0020099999 Diff : 0.0000000029 Float: 0.0030100048 Doubl: 0.0030100000 Cross: 0.0030099999 Diff : -0.0000000048 Float: 0.0040100124 Doubl: 0.0040100000 Cross: 0.0040099999 Diff : -0.0000000124 Float: 0.0050100200 Doubl: 0.0050100000 Cross: 0.0050099999 Diff : -0.0000000200 Float: 0.0060100276 Doubl: 0.0060100000 Cross: 0.0060099998 Diff : -0.0000000276 Float: 0.0070100352 Doubl: 0.0070100000 Cross: 0.0070099998 Diff : -0.0000000352 Float: 0.0080100335 Doubl: 0.0080100000 Cross: 0.0080099998 Diff : -0.0000000335
As you can see, after 100 iterations (first line) everything is OK. However, after 200 iterations, there’s already a difference (see “Diff: ” column) of -0.0000000005 between the double and float counters. Note that the doubles all end in 5 zeroes (as they should) while the float and cross variables end in some random assortment of digits. This is what first alerted the user to the problem.
This is one of the things I like about programming. If you’re not sure how something will work, it’s often easier to cook up a little test program and find out than to look around for an answer. Ultimately, it’s important to understand the difference between floats and doubles, but if the system doesn’t conform to IEEE standards, you might have to test it to find out anyway.
The moral of the story is: use doubles when you’re doing a lot of iterations with a counter. Don’t use floating-point values as increments at all. (See Rob’s comment for the correction).
ugh, of don’t use doubles as a counter at all. Here’s what I recommend:
double inc = 1.e-5;
for (int iter=0; iter < 900; iter++) {
double dnumber = inc*iter;
}
this form will have the minimum amount of round-off error possible.
Here’s some matlab code to prove my point:
n=10000000;
h=1/(n-1);
a = zeros(n,1);
b=[0:n-1]’*h;
for i=2:n
a(i) = a(i-1)+h;
end
error = a-b;
plot([0:n-1]’, error);
title ‘Error a_i-b_i, a_i=a_{i-1}+h, b_i=i*h’
xlabel ‘i’
ylabel ‘error’
This code has been written so that the final value for a and b is 1. Therefore, the error in this result will give you an idea of how many digits of error you can expect for different values of n.
Doubles have about 16 digits of precision. After 10 million iterations, you’ve lost 6 digits of precision. In floating point arithmetic, you will have lost almost all your digits of precision.
Here’s the plot:
I am getting error here for the following code i hope i will get proper guidance here to remove my error ..
the error is
ani@ani-desktop:~/Desktop/executedfiles$ javac retrievalGabor.java
retrievalGabor.java:54: incompatible types
found : double[]
required: float[]
query= new double[] {0.1544465f ,0.54298925f ,0.73340774f ,0.017640665f ,0.085013896f ,0.095653236f ,0.0 ,0.4047619f ,0.36394557f ,0.21768707f ,0.09863946f ,0.088435374f ,0.08163265f ,0.06462585f ,0.05782313f ,0.27210885f ,0.26870748f ,0.10204082f ,0.0 ,0.010204081f ,0.0034013605f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.023809524f ,0.010204081f ,0.0 ,0.0 ,0.17687075f ,0.32312927f ,0.0 ,0.0 ,0.09863946f ,0.6632653f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.013605442f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.08163265f ,0.0 ,0.0 ,0.0 ,0.49659863f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.010204081f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0};
^
1 error
code here:
//The alignment of the features is assumed to be AVG, VAR, CSD, GABOR
import java.io.*;
import javax.imageio.*;
import java.awt.*;
import java.awt.image.*;
class retrievalGabor
{
public static void main(String args[])
{
if(args.length!=6 && args.length!=7)
{
System.out.println(“The parameters are {Query Image} {Query Gabor file} {Features File} {Gabor Text File} [blocks] {Relevant Features} {EPS directory}”);
System.exit(0);
}
int BLOCKS=1, FEATURES=15, SCALE=3, ORIENTATION=4;
BufferedImage bi, bi1;
GraphicsEnvironment ge= GraphicsEnvironment.getLocalGraphicsEnvironment();
GraphicsDevice gd=ge.getDefaultScreenDevice();
GraphicsConfiguration gc=gd.getDefaultConfiguration();
Graphics2D g2D=null;
ColorModel cm=null;
DataInputStream dis, dis1;
BufferedReader br, br1;
float features[][], diff[], query[], tmp=0;
String names[], tmps=null, tokens[];
int images=0, wt[], bitcount=0;
char ch;
try{
bi=ImageIO.read(new File(args[0]));
bi1=gc.createCompatibleImage(bi.getWidth(null), bi.getHeight(null));
g2D=bi1.createGraphics();
g2D.drawImage(bi, null, 0, 0);
bi=bi1;
dis=new DataInputStream(new FileInputStream(args[2]));//opening the features file
br=new BufferedReader(new FileReader(args[3]));//opening the texture features file
if(args.length==7)
{
BLOCKS=Integer.parseInt(args[4]);
dis1=new DataInputStream(new FileInputStream(args[5]));//opening the relevant features file
}
else
dis1=new DataInputStream(new FileInputStream(args[4]));//opening the relevant features file
//query=myFinalToolkit.computeColorSampleImage(bi, BLOCKS, SCALE*ORIENTATION);
query= new double[] {0.1544465f ,0.54298925f ,0.73340774f ,0.017640665f ,0.085013896f ,0.095653236f ,0.0 ,0.4047619f ,0.36394557f ,0.21768707f ,0.09863946f ,0.088435374f ,0.08163265f ,0.06462585f ,0.05782313f ,0.27210885f ,0.26870748f ,0.10204082f ,0.0 ,0.010204081f ,0.0034013605f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.023809524f ,0.010204081f ,0.0 ,0.0 ,0.17687075f ,0.32312927f ,0.0 ,0.0 ,0.09863946f ,0.6632653f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.013605442f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.08163265f ,0.0 ,0.0 ,0.0 ,0.49659863f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.010204081f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0};
br1=new BufferedReader(new FileReader(args[1]));//opening the Query texture file
tokens=(br1.readLine()).split(” “);
for(int i=3;i<2*BLOCKS*SCALE*ORIENTATION;i++)
query[BLOCKS*6+120+i-3]=Float.parseFloat(tokens[i]);
br1.close();
//reading the features file
images=dis.readInt();
features=new float[images][BLOCKS*(6+SCALE*ORIENTATION*2)+120];
names=new String[images];
for(int i=0;i<images;i++)
{
names[i]=new String();
while((ch=dis.readChar())!=’\n’)
names[i]+=ch;
for(int j=0; j<BLOCKS*6+120;j++)
features[i][j]=dis.readFloat();
dis.readChar();
tokens=(br.readLine()).split(” “);
for(int j=3;j<BLOCKS*2*SCALE*ORIENTATION;j++)
features[i][BLOCKS*6+120+j-3]=Float.parseFloat(tokens[j]);
}
dis.close();
br.close();
//reading the feature relevancy
wt=new int[BLOCKS*(FEATURES-1)+1];
for(int i=0;i<BLOCKS*(FEATURES-1)+1;i++)
{
wt[i]=dis1.readInt();
System.out.print(wt[i]+” “);
}
System.out.println();
dis1.close();
diff=new float[images];
for(int i=0;i<images;i++)
{
//average and variance in color
for(int j=0;j<BLOCKS;j++)
{
for(int k=0;k<2;k++)
{
tmp=0;
if(wt[j*(FEATURES-1)+k]==1)
{
for(int l=0;l<3;l++)
tmp+=Math.abs(query[j*6+k*3+l]-features[i][j*6+k*3+l]);
diff[i]+=tmp/3;
}
}
}
//CSD
tmp=0;
if(wt[BLOCKS*(FEATURES-1)]==1)
{
for(int k=0;k<120;k++)
tmp+=Math.abs(query[BLOCKS*6+k]-features[i][BLOCKS*6+k]);
diff[i]+=tmp/120;
}
//texture features
for(int j=0;j<BLOCKS;j++)
for(int k=0;k<SCALE*ORIENTATION;k++)
{
tmp=0;
if(wt[j*(FEATURES-1)+2+k]==1)
for(int l=0;l<2;l++)
{
tmp+=Math.abs(query[BLOCKS*6+120+j*SCALE*ORIENTATION+2*k+l]-features[i][BLOCKS*6+120+j*SCALE*ORIENTATION+2*k+l]);
}
diff[i]+=tmp/2;
}
}
//sort the entries in the sorted order
for(int i=0;i<images-1;i++)
for(int j=0;j<images-i-1;j++)
{
if(diff[j+1]<diff[j])
{
tmp=diff[j];
diff[j]=diff[j+1];
diff[j+1]=tmp;
tmps=names[j];
names[j]=names[j+1];
names[j+1]=tmps;
tmps=null;
}
}
/* //sort the entries in the sorted order by name
for(int i=0;i<29;i++)
for(int j=0;j0)
{
tmp=diff[j];
diff[j]=diff[j+1];
diff[j+1]=tmp;
tmps=names[j];
names[j]=names[j+1];
names[j+1]=tmps;
tmps=null;
}
}
*/
Runtime rt=Runtime.getRuntime();
String arr[]=new String[3], dir[]=new String[2];
dir[0]=new String(“/bin/mkdir”);
if(args.length==6)
dir[1]=new String(“/tmp/”+args[5]+args[0].substring(args[0].lastIndexOf(‘/’)+1, args[0].lastIndexOf(‘.’)));
else
dir[1]=new String(“/tmp/”+args[6]+args[0].substring(args[0].lastIndexOf(‘/’)+1, args[0].lastIndexOf(‘.’)));
rt.exec(dir);
arr[0]=new String(“/bin/cp”);
for(int i=0;i<30;i++)
{
System.out.println(names[i]+” “+diff[i]);
arr[1]=new String(args[0].substring(0,args[0].lastIndexOf(“/”)+1)+names[i]);
if(i<10)
arr[2]=new String(dir[1]+”/0″+i+”.jpg”);
else
arr[2]=new String(dir[1]+”/”+i+”.jpg”);
rt.exec(arr);
}
}
catch(Exception e)
{
e.printStackTrace();
System.exit(0);
}
}
}
Brilliant. Thank you for elaborating it.
Pingback: Ever wonder the difference between double vs float
Pingback: Tim's blag
Great example..!