Float vs. Double

Someone in the lab recently came to me asking for help in diagnosing a strange problem. The time output from their program was drifting inexplicably — that is, they were adding a certain increment every iteration, but the time was changing by some value slightly different from that increment. It turns out that this was because they were using single-precision floating-point (a.k.a “float”) variables. The problem did not show up right away. In fact, it would not have been an issue were they not adding an increment so very many times.

To diagnose this problem, (because I haven’t dealt with this issue in a long while), I wrote this little piece of code:

#include <stdio.h>

int main(void){

  float float_v, fdt;
  double double_v, cross_v, ddt;
  int i, found;
  
  float_v = 0;
  double_v = 0;
  cross_v = 0;
  found = 0;

  ddt = 1e-5;
  fdt = 1e-5;
  for(i=0; i< 10000; i++){
    double_v += ddt;
    float_v += fdt;
    cross_v += fdt;
     if(i%100 == 0){ 
       printf("Float: %5.10f ", float_v);
       printf("Doubl: %5.10f ", double_v);
       printf("Cross: %5.10f ", cross_v); 
       printf("Diff : %5.10f\n", double_v - float_v);
     } 

/* use this to find how many iterations before it screws up
    if(double_v - float_v > 1e-8 && !found){
      printf("Found discrepancy of %5.10f at step %d!\n", \
               double_v - float_v, i);
      found = 1;
    }
*/
  }

  return 0;
}

(You can download the code here: add_test.c)

This code tests three combinations:

  1. Float counter with float increment
  2. Double counter with double increment
  3. Double counter with float increment

The little commented section (starts with /* and ends with */) will (if uncommented) stop the program when the difference between the all-double value and the all-float value exceeds 10-8. On Mac OS X on a 64-bit Intel architecture, this happens after 912 steps:


Found discrepancy of 0.0000000101 at step 912!

Here’s the output from 900 iterations (printed every 100):

Float: 0.0000100000 Doubl: 0.0000100000 Cross: 0.0000100000 Diff : 0.0000000000
Float: 0.0010100005 Doubl: 0.0010100000 Cross: 0.0010100000 Diff : -0.0000000005
Float: 0.0020099971 Doubl: 0.0020100000 Cross: 0.0020099999 Diff : 0.0000000029
Float: 0.0030100048 Doubl: 0.0030100000 Cross: 0.0030099999 Diff : -0.0000000048
Float: 0.0040100124 Doubl: 0.0040100000 Cross: 0.0040099999 Diff : -0.0000000124
Float: 0.0050100200 Doubl: 0.0050100000 Cross: 0.0050099999 Diff : -0.0000000200
Float: 0.0060100276 Doubl: 0.0060100000 Cross: 0.0060099998 Diff : -0.0000000276
Float: 0.0070100352 Doubl: 0.0070100000 Cross: 0.0070099998 Diff : -0.0000000352
Float: 0.0080100335 Doubl: 0.0080100000 Cross: 0.0080099998 Diff : -0.0000000335

As you can see, after 100 iterations (first line) everything is OK. However, after 200 iterations, there’s already a difference (see “Diff: ” column) of -0.0000000005 between the double and float counters. Note that the doubles all end in 5 zeroes (as they should) while the float and cross variables end in some random assortment of digits. This is what first alerted the user to the problem.

This is one of the things I like about programming. If you’re not sure how something will work, it’s often easier to cook up a little test program and find out than to look around for an answer. Ultimately, it’s important to understand the difference between floats and doubles, but if the system doesn’t conform to IEEE standards, you might have to test it to find out anyway.

The moral of the story is: use doubles when you’re doing a lot of iterations with a counter. Don’t use floating-point values as increments at all. (See Rob’s comment for the correction).

8 thoughts on “Float vs. Double

  1. Rob

    ugh, of don’t use doubles as a counter at all. Here’s what I recommend:

    double inc = 1.e-5;
    for (int iter=0; iter < 900; iter++) {
    double dnumber = inc*iter;
    }

  2. Rob Blake

    Here’s some matlab code to prove my point:

    n=10000000;
    h=1/(n-1);
    a = zeros(n,1);
    b=[0:n-1]’*h;
    for i=2:n
    a(i) = a(i-1)+h;
    end

    error = a-b;

    plot([0:n-1]’, error);
    title ‘Error a_i-b_i, a_i=a_{i-1}+h, b_i=i*h’
    xlabel ‘i’
    ylabel ‘error’

    This code has been written so that the final value for a and b is 1. Therefore, the error in this result will give you an idea of how many digits of error you can expect for different values of n.

    Doubles have about 16 digits of precision. After 10 million iterations, you’ve lost 6 digits of precision. In floating point arithmetic, you will have lost almost all your digits of precision.

    Here’s the plot:
    float iteration error graph

  3. anitha

    I am getting error here for the following code i hope i will get proper guidance here to remove my error ..
    the error is

    ani@ani-desktop:~/Desktop/executedfiles$ javac retrievalGabor.java
    retrievalGabor.java:54: incompatible types
    found : double[]
    required: float[]
    query= new double[] {0.1544465f ,0.54298925f ,0.73340774f ,0.017640665f ,0.085013896f ,0.095653236f ,0.0 ,0.4047619f ,0.36394557f ,0.21768707f ,0.09863946f ,0.088435374f ,0.08163265f ,0.06462585f ,0.05782313f ,0.27210885f ,0.26870748f ,0.10204082f ,0.0 ,0.010204081f ,0.0034013605f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.023809524f ,0.010204081f ,0.0 ,0.0 ,0.17687075f ,0.32312927f ,0.0 ,0.0 ,0.09863946f ,0.6632653f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.013605442f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.08163265f ,0.0 ,0.0 ,0.0 ,0.49659863f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.010204081f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0};
    ^
    1 error

    code here:

    //The alignment of the features is assumed to be AVG, VAR, CSD, GABOR

    import java.io.*;
    import javax.imageio.*;
    import java.awt.*;
    import java.awt.image.*;

    class retrievalGabor
    {
    public static void main(String args[])
    {
    if(args.length!=6 && args.length!=7)
    {
    System.out.println(“The parameters are {Query Image} {Query Gabor file} {Features File} {Gabor Text File} [blocks] {Relevant Features} {EPS directory}”);
    System.exit(0);
    }

    int BLOCKS=1, FEATURES=15, SCALE=3, ORIENTATION=4;
    BufferedImage bi, bi1;
    GraphicsEnvironment ge= GraphicsEnvironment.getLocalGraphicsEnvironment();
    GraphicsDevice gd=ge.getDefaultScreenDevice();
    GraphicsConfiguration gc=gd.getDefaultConfiguration();
    Graphics2D g2D=null;
    ColorModel cm=null;
    DataInputStream dis, dis1;
    BufferedReader br, br1;

    float features[][], diff[], query[], tmp=0;
    String names[], tmps=null, tokens[];
    int images=0, wt[], bitcount=0;
    char ch;

    try{
    bi=ImageIO.read(new File(args[0]));
    bi1=gc.createCompatibleImage(bi.getWidth(null), bi.getHeight(null));
    g2D=bi1.createGraphics();
    g2D.drawImage(bi, null, 0, 0);
    bi=bi1;

    dis=new DataInputStream(new FileInputStream(args[2]));//opening the features file
    br=new BufferedReader(new FileReader(args[3]));//opening the texture features file

    if(args.length==7)
    {
    BLOCKS=Integer.parseInt(args[4]);
    dis1=new DataInputStream(new FileInputStream(args[5]));//opening the relevant features file
    }
    else
    dis1=new DataInputStream(new FileInputStream(args[4]));//opening the relevant features file

    //query=myFinalToolkit.computeColorSampleImage(bi, BLOCKS, SCALE*ORIENTATION);
    query= new double[] {0.1544465f ,0.54298925f ,0.73340774f ,0.017640665f ,0.085013896f ,0.095653236f ,0.0 ,0.4047619f ,0.36394557f ,0.21768707f ,0.09863946f ,0.088435374f ,0.08163265f ,0.06462585f ,0.05782313f ,0.27210885f ,0.26870748f ,0.10204082f ,0.0 ,0.010204081f ,0.0034013605f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.023809524f ,0.010204081f ,0.0 ,0.0 ,0.17687075f ,0.32312927f ,0.0 ,0.0 ,0.09863946f ,0.6632653f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.013605442f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.08163265f ,0.0 ,0.0 ,0.0 ,0.49659863f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.010204081f ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0 ,0.0};

    br1=new BufferedReader(new FileReader(args[1]));//opening the Query texture file
    tokens=(br1.readLine()).split(” “);
    for(int i=3;i<2*BLOCKS*SCALE*ORIENTATION;i++)
    query[BLOCKS*6+120+i-3]=Float.parseFloat(tokens[i]);
    br1.close();

    //reading the features file
    images=dis.readInt();
    features=new float[images][BLOCKS*(6+SCALE*ORIENTATION*2)+120];
    names=new String[images];

    for(int i=0;i<images;i++)
    {
    names[i]=new String();
    while((ch=dis.readChar())!=’\n’)
    names[i]+=ch;

    for(int j=0; j<BLOCKS*6+120;j++)
    features[i][j]=dis.readFloat();
    dis.readChar();

    tokens=(br.readLine()).split(” “);
    for(int j=3;j<BLOCKS*2*SCALE*ORIENTATION;j++)
    features[i][BLOCKS*6+120+j-3]=Float.parseFloat(tokens[j]);
    }
    dis.close();
    br.close();

    //reading the feature relevancy
    wt=new int[BLOCKS*(FEATURES-1)+1];
    for(int i=0;i<BLOCKS*(FEATURES-1)+1;i++)
    {
    wt[i]=dis1.readInt();
    System.out.print(wt[i]+” “);
    }
    System.out.println();
    dis1.close();

    diff=new float[images];
    for(int i=0;i<images;i++)
    {
    //average and variance in color
    for(int j=0;j<BLOCKS;j++)
    {
    for(int k=0;k<2;k++)
    {
    tmp=0;
    if(wt[j*(FEATURES-1)+k]==1)
    {
    for(int l=0;l<3;l++)
    tmp+=Math.abs(query[j*6+k*3+l]-features[i][j*6+k*3+l]);

    diff[i]+=tmp/3;
    }
    }
    }

    //CSD
    tmp=0;
    if(wt[BLOCKS*(FEATURES-1)]==1)
    {
    for(int k=0;k<120;k++)
    tmp+=Math.abs(query[BLOCKS*6+k]-features[i][BLOCKS*6+k]);

    diff[i]+=tmp/120;
    }

    //texture features
    for(int j=0;j<BLOCKS;j++)
    for(int k=0;k<SCALE*ORIENTATION;k++)
    {
    tmp=0;
    if(wt[j*(FEATURES-1)+2+k]==1)
    for(int l=0;l<2;l++)
    {
    tmp+=Math.abs(query[BLOCKS*6+120+j*SCALE*ORIENTATION+2*k+l]-features[i][BLOCKS*6+120+j*SCALE*ORIENTATION+2*k+l]);
    }
    diff[i]+=tmp/2;
    }

    }

    //sort the entries in the sorted order
    for(int i=0;i<images-1;i++)
    for(int j=0;j<images-i-1;j++)
    {
    if(diff[j+1]<diff[j])
    {
    tmp=diff[j];
    diff[j]=diff[j+1];
    diff[j+1]=tmp;

    tmps=names[j];
    names[j]=names[j+1];
    names[j+1]=tmps;

    tmps=null;
    }
    }

    /* //sort the entries in the sorted order by name
    for(int i=0;i<29;i++)
    for(int j=0;j0)
    {
    tmp=diff[j];
    diff[j]=diff[j+1];
    diff[j+1]=tmp;

    tmps=names[j];
    names[j]=names[j+1];
    names[j+1]=tmps;

    tmps=null;
    }
    }
    */

    Runtime rt=Runtime.getRuntime();
    String arr[]=new String[3], dir[]=new String[2];

    dir[0]=new String(“/bin/mkdir”);
    if(args.length==6)
    dir[1]=new String(“/tmp/”+args[5]+args[0].substring(args[0].lastIndexOf(‘/’)+1, args[0].lastIndexOf(‘.’)));
    else
    dir[1]=new String(“/tmp/”+args[6]+args[0].substring(args[0].lastIndexOf(‘/’)+1, args[0].lastIndexOf(‘.’)));

    rt.exec(dir);

    arr[0]=new String(“/bin/cp”);
    for(int i=0;i<30;i++)
    {
    System.out.println(names[i]+” “+diff[i]);
    arr[1]=new String(args[0].substring(0,args[0].lastIndexOf(“/”)+1)+names[i]);

    if(i<10)
    arr[2]=new String(dir[1]+”/0″+i+”.jpg”);
    else
    arr[2]=new String(dir[1]+”/”+i+”.jpg”);
    rt.exec(arr);
    }

    }
    catch(Exception e)
    {
    e.printStackTrace();
    System.exit(0);
    }
    }
    }

  4. Pingback: Ever wonder the difference between double vs float

  5. Pingback: Tim's blag

Comments are closed.