strtonum in awk causes the values to lose precision
I have a text file in the following format.
1 0x5212cb03ca115ac0 0x3665fb5f1ac1
2 0x5212cb03ca115cc0 0x3665fb5f1ac7
3 0x5212cb03ca115ea0 0x3665fb5f1acd
4 0x5212cb03ca1160c0 0x3665fb5f1ad3
5 0x5212cb03ca1162a0 0x3665fb5f1ad9
6 0x5212cb03ca1164c0 0x3665fb5f1ade
7 0x5212cb03ca1166a0 0x3665fb5f1ae4
8 0x5212cb03ca1168a0 0x3665fb5f1aea
9 0x5212cb03ca116aa0 0x3665fb5f1af0
10 0x5212cb03ca116ca0 0x3665fb5f1af6
awk '{print $1 " "strtonum($2)-0x5212cb03ca115ac0 "
"strtonum($3)-0x3665fb5f1ac1 }' output.txt
The output that I get is given below.
1 0 0
2 1024 6
3 2048 12
4 2048 18
5 2048 24
6 3072 29
7 4096 35
8 4096 41
9 4096 47
10 5120 53
If you see the values in column 2 has some values repeating
themselves.(2048 and 4096).This is caused due to loss of precision when
using strtonum
Can someone suggest some method to achieve the same but avoid this loss of
precision.
No comments:
Post a Comment